data-analysis

これはレビュー時に保存された抜粋です。完全で最新の内容は外部ソースを確認してください。

---
name: data-analysis
description: Analyze datasets to answer defined questions through statistical methods, trend identification, hypothesis testing, and correlation analysis. Use when the user needs evidence-backed findings or decisions from data; use exploratory-data-analysis instead for open-ended first-pass profiling before questions are defined.
license: MIT
metadata:
  author: awesome-ai-agent-skills
  version: 1.0.0
---

# Data Analysis

This skill enables an AI agent to perform rigorous statistical analysis on structured datasets. The agent loads data, computes descriptive and inferential statistics, identifies trends and correlations, tests hypotheses, and produces actionable insights. It supports CSV, Excel, Parquet, and JSON inputs and leverages pandas, scipy, and statsmodels for analysis.

## Workflow

1. **Load and profile the data.** Read the dataset into a pandas DataFrame and inspect its shape, column types, and memory usage. Display the first and last rows to confirm the data loaded correctly. Check for obvious structural issues such as shifted columns or encoding problems.

2. **Compute descriptive statistics.** Generate summary statistics for all numeric columns including mean, median, standard deviation, skewness, and kurtosis. For categorical columns, compute value counts and mode. This step establishes a baseline understanding of each variable's distribution.

3. **Identify trends and patterns.** Apply rolling averages, percentage changes, and seasonal decomposition to time-indexed data. For non-temporal data, use group-by aggregations and pivot tables to surface patterns across categories. Flag any monotonic trends or cyclical behavior.

4. **Perform correlation and hypothesis testing.** Calculate Pearson and Spearman correlation matrices to quantify relationships between variables. Conduct hypothesis tests (t-tests, chi-square, ANOVA) where appropriate to determine statistical significance. Report p-values and confidence intervals alongside effect sizes.

5. **Detect anomalies and outliers.** Use the IQR method and z-scores to identify data points that deviate significantly from the norm. Cross-reference outliers with domain context to determine whether they represent errors, rare events, or meaningful signals.

6. **Synthesize findings into a report.** Summarize the key insights in plain language, supported by specific numbers. Rank findings by business impact or statistical significance. Include limitations and caveats such as sample size constraints or confounding variables.

## Supported Technologies

- **pandas** — data loading, manipulation, and aggregation
- **scipy.stats** — hypothesis testing, statistical distributions
- **statsmodels** — time-series decomposition, regression analysis
- **numpy** — numerical computations

## Usage

Provide the agent with a file path to the dataset and a description of the analysis goals. Optionally specify which columns to focus on, the significance level for hypothesis tests (default alpha=0.05), and whether time-series methods should be applied.

## Examples

### Example 1: Sales CSV analysis with pandas

```python
import pandas as pd
from scipy import stats

# Load the dataset
df = pd.read_csv("sales_2024.csv", parse_dates=["order_date"])

# Descriptive statistics
print(df[["revenue", "units_sold", "discount"]].describe())
#          revenue  units_sold  discount
# count   8450.00     8450.00   8450.00
# mean     312.45       4.12      0.08
# std      189.73       2.87      0.05
# min       12.00       1.00      0.00
# max     2450.00      47.00      0.35

# Correlation analysis
corr = df[["revenue", "units_sold", "discount"]].corr(method="pearson")
print(corr)
#             revenue  units_sold  discount
# revenue       1.000       0.847    -0.213
# units_sold    0.847       1.000    -0.089
# discount     -0.213      -0.089     1.000

# Hypothesis test: do discounted orders produce higher revenue?
discounted = df[df["discount"] > 0]["revenue"]
full_price