statistical-analysis

# Statistical Analysis

## Overview

Statistical analysis is a systematic process for testing hypotheses and quantifying relationships. Conduct hypothesis tests (t-test, ANOVA, chi-square), regression, correlation, and Bayesian analyses with assumption checks and APA reporting. Apply this skill for academic research.

## When to Use This Skill

This skill should be used when:
- Conducting statistical hypothesis tests (t-tests, ANOVA, chi-square)
- Performing regression or correlation analyses
- Running Bayesian statistical analyses
- Checking statistical assumptions and diagnostics
- Calculating effect sizes and conducting power analyses
- Reporting statistical results in APA format
- Analyzing experimental or observational data for research

---

## Core Capabilities

### 1. Test Selection and Planning
- Choose appropriate statistical tests based on research questions and data characteristics
- Conduct a priori power analyses to determine required sample sizes
- Plan analysis strategies including multiple comparison corrections

### 2. Assumption Checking
- Automatically verify all relevant assumptions before running tests
- Provide diagnostic visualizations (Q-Q plots, residual plots, box plots)
- Recommend remedial actions when assumptions are violated

### 3. Statistical Testing
- Hypothesis testing: t-tests, ANOVA, chi-square, non-parametric alternatives
- Regression: linear, multiple, logistic, with diagnostics
- Correlations: Pearson, Spearman, with confidence intervals
- Bayesian alternatives: Bayesian t-tests, ANOVA, regression with Bayes Factors

### 4. Effect Sizes and Interpretation
- Calculate and interpret appropriate effect sizes for all analyses
- Provide confidence intervals for effect estimates
- Distinguish statistical from practical significance

### 5. Professional Reporting
- Generate APA-style statistical reports
- Create publication-ready figures and tables
- Provide complete interpretation with all required statistics

---

## Workflow Decision Tree

Use this decision tree to determine your analysis path:

```
START
│
├─ Need to SELECT a statistical test?
│  └─ YES → See "Test Selection Guide"
│  └─ NO → Continue
│
├─ Ready to check ASSUMPTIONS?
│  └─ YES → See "Assumption Checking"
│  └─ NO → Continue
│
├─ Ready to run ANALYSIS?
│  └─ YES → See "Running Statistical Tests"
│  └─ NO → Continue
│
└─ Need to REPORT results?
   └─ YES → See "Reporting Results"
```

---

## Test Selection Guide

### Quick Reference: Choosing the Right Test

Use `references/test_selection_guide.md` for comprehensive guidance. Quick reference:

**Comparing Two Groups:**
- Independent, continuous, normal → Independent t-test
- Independent, continuous, non-normal → Mann-Whitney U test
- Paired, continuous, normal → Paired t-test
- Paired, continuous, non-normal → Wilcoxon signed-rank test
- Binary outcome → Chi-square or Fisher's exact test

**Comparing 3+ Groups:**
- Independent, continuous, normal → One-way ANOVA
- Independent, continuous, non-normal → Kruskal-Wallis test
- Paired, continuous, normal → Repeated measures ANOVA
- Paired, continuous, non-normal → Friedman test

**Relationships:**
- Two continuous variables → Pearson (normal) or Spearman correlation (non-normal)
- Continuous outcome with predictor(s) → Linear regression
- Binary outcome with predictor(s) → Logistic regression

**Bayesian Alternatives:**
All tests have Bayesian versions that provide:
- Direct probability statements about hypotheses
- Bayes Factors quantifying evidence
- Ability to support null hypothesis
- See `references/bayesian_statistics.md`

---

## Assumption Checking

### Systematic Assumption Verification

**ALWAYS check assumptions before interpreting test results.**

Use the provided `scripts/assumption_checks.py` module for automated checking:

```python
from scripts.assumption_checks import comprehensive_assumption_check

# Comprehensive check with visualizations
results = comprehensive_assumption_check(
    data=df,
    value_col='score',
    group_col='group',  # Optional: for group comparisons
    alpha=0.05
)
```

This performs:
1. **Outlier detection** (IQR and z-score methods)
2. **Normality testing** (Shapiro-Wilk test + Q-Q plots)
3. **Homogeneity of variance** (Levene's test + box plots)
4. **Interpretation and recommendations**

### Individual Assumption Checks

For targeted checks, use individual functions:

```python
from scripts.assumption_checks import (
    check_normality,
    check_normality_per_group,
    check_homogeneity_of_variance,
    check_linearity,
    detect_outliers
)

# Example: Check normality with visualization
result = check_normality(
    data=df['score'],
    name='Test Score',
    alpha=0.05,
    plot=True
)
print(result['interpretation'])
print(result['recommendation'])
```

### What to Do When Assumptions Are Violated

**Normality violated:**
- Mild violation + n > 30 per group → Proceed with parametric test (robust)
- Moderate violation → Use non-parametric alternative
- Severe violation → Transform data or use non-parametric test

**Homogeneity of variance violated:**
- For t-test → Use Welch's t-test
- For ANOVA → Use Welch's ANOVA or Brown-Forsythe ANOVA
- For regression → Use robust standard errors or weighted least squares

**Linearity violated (regression):**
- Add polynomial terms
- Transform variables
- Use non-linear models or GAM

See `references/assumptions_and_diagnostics.md` for comprehensive guidance.

---

## Running Statistical Tests

### Python Libraries

Primary libraries for statistical analysis:
- **scipy.stats**: Core statistical tests
- **statsmodels**: Advanced regression and diagnostics
- **pingouin**: User-friendly statistical testing with effect sizes
- **pymc**: Bayesian statistical modeling
- **arviz**: Bayesian visualization and diagnostics

### Example Analyses

#### T-Test with Complete Reporting

```python
import pingouin as pg
import numpy as np

# Run independent t-test
result = pg.ttest(group_a, group_b, correction='auto')

# Extract results
t_stat = result['T'].values[0]
df = result['dof'].values[0]
p_value = result['p-val'].values[0]
cohens_d = result['cohen-d'].values[0]
ci_lower = result['CI95%'].values[0][0]
ci_upper = result['CI95%'].values[0][1]

# Report
print(f"t({df:.0f}) = {t_stat:.2f}, p = {p_value:.3f}")
print(f"Cohen's d = {cohens_d:.2f}, 95% CI [{ci_lower:.2f}, {ci_upper:.2f}]")
```

#### ANOVA with Post-Hoc Tests

```python
import pingouin as pg

# One-way ANOVA
aov = pg.anova(dv='score', between='group', data=df, detailed=True)
print(aov)

# If significant, conduct post-hoc tests
if aov['p-unc'].values[0] < 0.05:
    posthoc = pg.pairwise_tukey(dv='score', between='group', data=df)
    print(posthoc)

# Effect size
eta_squared = aov['np2'].values[0]  # Partial eta-squared
print(f"Partial η² = {eta_squared:.3f}")
```

#### Linear Regression with Diagnostics

```python
import statsmodels.api as sm
from statsmodels.stats.outliers_influence import variance_inflation_factor

# Fit model
X = sm.add_constant(X_predictors)  # Add intercept
model = sm.OLS(y, X).fit()

# Summary
print(model.summary())

# Check multicollinearity (VIF)
vif_data = pd.DataFrame()
vif_data["Variable"] = X.columns
vif_data["VIF"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]
print(vif_data)

# Check assumptions
residuals = model.resid
fitted = model.fittedvalues

# Residual plots
import matplotlib.pyplot as plt
fig, axes = plt.subplots(2, 2, figsize=(12, 10))

# Residuals vs fitted
axes[0, 0].scatter(fitted, residuals, alpha=0.6)
axes[0, 0].axhline(y=0, color='r', linestyle='--')
axes[0, 0].set_xlabel('Fitted values')
axes[0, 0].set_ylabel('Residuals')
axes[0, 0].set_title('Residuals vs Fitted')

# Q-Q plot
from scipy import stats
stats.probplot(residuals, dist="norm", plot=axes[0, 1])
axes[0, 1].set_title('Normal Q-Q')

# Scale-Location
axes[1, 0].scatter(fitted, np.sqrt(np.abs(residuals / residuals.std())), alpha=0.6)
axes[1, 0].set_xlabel('Fitted values')
axes[1, 0].set_ylabel('√|Standardized residuals|')
axes[1, 0].set_title('Scale-Location')

# Residuals histogram
axes[1, 1].hist(residuals, bins=20, edgecolor='black', alpha=0.7)
axes[1, 1].set_xlabel('Residuals')
axes[1, 1].set_ylabel('Frequency')
axes[1, 1].set_title('Histogram of Residuals')

plt.tight_layout()
plt.show()
```

#### Bayesian T-Test

```python
import pymc as pm
import arviz as az
import numpy as np

with pm.Model() as model:
    # Priors
    mu1 = pm.Normal('mu_group1', mu=0, sigma=10)
    mu2 = pm.Normal('mu_group2', mu=0, sigma=10)
    sigma = pm.HalfNormal('sigma', sigma=10)

    # Likelihood
    y1 = pm.Normal('y1', mu=mu1, sigma=sigma, observed=group_a)
    y2 = pm.Normal('y2', mu=mu2, sigma=sigma, observed=group_b)

    # Derived quantity
    diff = pm.Deterministic('difference', mu1 - mu2)

    # Sample
    trace = pm.sample(2000, tune=1000, return_inferencedata=True)

# Summarize
print(az.summary(trace, var_names=['difference']))

# Probability that group1 > group2
prob_greater = np.mean(trace.posterior['difference'].values > 0)
print(f"P(μ₁ > μ₂ | data) = {prob_greater:.3f}")

# Plot posterior
az.plot_posterior(trace, var_names=['difference'], ref_val=0)
```

---

## Effect Sizes

### Always Calculate Effect Sizes

**Effect sizes quantify magnitude, while p-values only indicate existence of an effect.**

See `references/effect_sizes_and_power.md` for comprehensive guidance.

### Quick Reference: Common Effect Sizes

| Test | Effect Size | Small | Medium | Large |
|------|-------------|-------|--------|-------|
| T-test | Cohen's d | 0.20 | 0.50 | 0.80 |
| ANOVA | η²_p | 0.01 | 0.06 | 0.14 |
| Correlation | r | 0.10 | 0.30 | 0.50 |
| Regression | R² | 0.02 | 0.13 | 0.26 |
| Chi-square | Cramér's V | 0.07 | 0.21 | 0.35 |

**Important**: Benchmarks are guidelines. Context matters!

### Calculating Effect Sizes

Most effect sizes are automatically calculated by pingouin:

```python
# T-test returns Cohen's d
result = pg.ttest(x, y)
d = result['cohen-d'].values[0]

# ANOVA returns partial eta-squared
aov = pg.anova(dv='score', between='group', data=df)
eta_p2 = aov['np2'].values[0]

# Correlation: r is already an effect size
corr = pg.corr(x, y)
r = corr['r'].values[0]
```

### Confidence Intervals for Effect Sizes

Always report CIs to show precision:

```python
from pingouin import compute_effsize_from_t

# For t-test
d, ci = compute_effsize_from_t(
    t_statistic,
    nx=len(group1),
    ny=len(group2),
    eftype='cohen'
)
print(f"d = {d:.2f}, 95% CI [{ci[0]:.2f}, {ci[1]:.2f}]")
```

---

## Power Analysis

### A Priori Power Analysis (Study Planning)

Determine required sample size before data collection:

```python
from statsmodels.stats.power import (
    tt_ind_solve_power,
    FTestAnovaPower
)

# T-test: What n is needed to detect d = 0.5?
n_required = tt_ind_solve_power(
    effect_size=0.5,
    alpha=0.05,
    power=0.80,
    ratio=1.0,
    alternative='two-sided'
)
print(f"Required n per group: {n_required:.0f}")

# ANOVA: What n is needed to detect f = 0.25?
anova_power = FTestAnovaPower()
n_per_group = anova_power.solve_power(
    effect_size=0.25,
    ngroups=3,
    alpha=0.05,
    power=0.80
)
print(f"Required n per group: {n_per_group:.0f}")
```

### Sensitivity Analysis (Post-Study)

Determine what effect size you could detect:

```python
# With n=50 per group, what effect could we detect?
detectable_d = tt_ind_solve_power(
    effect_size=None,  # Solve for this
    nobs1=50,
    alpha=0.05,
    power=0.80,
    ratio=1.0,
    alternative='two-sided'
)
print(f"Study could detect d ≥ {detectable_d:.2f}")
```

**Note**: Post-hoc power analysis (calculating power after study) is generally not recommended. Use sensitivity analysis instead.

See `references/effect_sizes_and_power.md` for detailed guidance.

---

## Reporting Results

### APA Style Statistical Reporting

Follow guidelines in `references/reporting_standards.md`.

### Essential Reporting Elements

1. **Descriptive statistics**: M, SD, n for all groups/variables
2. **Test statistics**: Test name, statistic, df, exact p-value
3. **Effect sizes**: With confidence intervals
4. **Assumption checks**: Which tests were done, results, actions taken
5. **All planned analyses**: Including non-significant findings

### Example Report Templates

#### Independent T-Test

```
Group A (n = 48, M = 75.2, SD = 8.5) scored significantly higher than
Group B (n = 52, M = 68.3, SD = 9.2), t(98) = 3.82, p < .001, d = 0.77,
95% CI [0.36, 1.18], two-tailed. Assumptions of normality (Shapiro-Wilk:
Group A W = 0.97, p = .18; Group B W = 0.96, p = .12) and homogeneity
of variance (Levene's F(1, 98) = 1.23, p = .27) were satisfied.
```

#### One-Way ANOVA

```
A one-way ANOVA revealed a significant main effect of treatment condition
on test scores, F(2, 147) = 8.45, p < .001, η²_p = .10. Post hoc
comparisons using Tukey's HSD indicated that Condition A (M = 78.2,
SD = 7.3) scored significantly higher than Condition B (M = 71.5,
SD = 8.1, p = .002, d = 0.87) and Condition C (M = 70.1, SD = 7.9,
p < .001, d = 1.07). Conditions B and C did not differ significantly
(p = .52, d = 0.18).
```

#### Multiple Regression

```
Multiple linear regression was conducted to predict exam scores from
study hours, prior GPA, and attendance. The overall model was significant,
F(3, 146) = 45.2, p < .001, R² = .48, adjusted R² = .47. Study hours
(B = 1.80, SE = 0.31, β = .35, t = 5.78, p < .001, 95% CI [1.18, 2.42])
and prior GPA (B = 8.52, SE = 1.95, β = .28, t = 4.37, p < .001,
95% CI [4.66, 12.38]) were significant predictors, while attendance was
not (B = 0.15, SE = 0.12, β = .08, t = 1.25, p = .21, 95% CI [-0.09, 0.39]).
Multicollinearity was not a concern (all VIF < 1.5).
```

#### Bayesian Analysis

```
A Bayesian independent samples t-test was conducted using weakly
informative priors (Normal(0, 1) for mean difference). The posterior
distribution indicated that Group A scored higher than Group B
(M_diff = 6.8, 95% credible interval [3.2, 10.4]). The Bayes Factor
BF₁₀ = 45.3 provided very strong evidence for a difference between
groups, with a 99.8% posterior probability that Group A's mean exceeded
Group B's mean. Convergence diagnostics were satisfactory (all R̂ < 1.01,
ESS > 1000).
```

---

## Bayesian Statistics

### When to Use Bayesian Methods

Consider Bayesian approaches when:
- You have prior information to incorporate
- You want direct probability statements about hypotheses
- Sample size is small or planning sequential data collection
- You need to quantify evidence for the null hypothesis
- The model is complex (hierarchical, missing data)

See `references/bayesian_statistics.md` for comprehensive guidance on:
- Bayes' theorem and interpretation
- Prior specification (informative, weakly informative, non-informative)
- Bayesian hypothesis testing with Bayes Factors
- Credible intervals vs. confidence intervals
- Bayesian t-tests, ANOVA, regression, and hierarchical models
- Model convergence checking and posterior predictive checks

### Key Advantages

1. **Intuitive interpretation**: "Given the data, there is a 95% probability the parameter is in this interval"
2. **Evidence for null**: Can quantify support for no effect
3. **Flexible**: No p-hacking concerns; can analyze data as it arrives
4. **Uncertainty quantification**: Full posterior distribution

---

## Resources

This skill includes comprehensive reference materials:

### References Directory

- **test_selection_guide.md**: Decision tree for choosing appropriate statistical tests
- **assumptions_and_diagnostics.md**: Detailed guidance on checking and handling assumption violations
- **effect_sizes_and_power.md**: Calculating, interpreting, and reporting effect sizes; conducting power analyses
- **bayesian_statistics.md**: Complete guide to Bayesian analysis methods
- **reporting_standards.md**: APA-style reporting guidelines with examples

### Scripts Directory

- **assumption_checks.py**: Automated assumption checking with visualizations
  - `comprehensive_assumption_check()`: Complete workflow
  - `check_normality()`: Normality testing with Q-Q plots
  - `check_homogeneity_of_variance()`: Levene's test with box plots
  - `check_linearity()`: Regression linearity checks
  - `detect_outliers()`: IQR and z-score outlier detection

---

## Best Practices

1. **Pre-register analyses** when possible to distinguish confirmatory from exploratory
2. **Always check assumptions** before interpreting results
3. **Report effect sizes** with confidence intervals
4. **Report all planned analyses** including non-significant results
5. **Distinguish statistical from practical significance**
6. **Visualize data** before and after analysis
7. **Check diagnostics** for regression/ANOVA (residual plots, VIF, etc.)
8. **Conduct sensitivity analyses** to assess robustness
9. **Share data and code** for reproducibility
10. **Be transparent** about violations, transformations, and decisions

---

## Common Pitfalls to Avoid

1. **P-hacking**: Don't test multiple ways until something is significant
2. **HARKing**: Don't present exploratory findings as confirmatory
3. **Ignoring assumptions**: Check them and report violations
4. **Confusing significance with importance**: p < .05 ≠ meaningful effect
5. **Not reporting effect sizes**: Essential for interpretation
6. **Cherry-picking results**: Report all planned analyses
7. **Misinterpreting p-values**: They're NOT probability that hypothesis is true
8. **Multiple comparisons**: Correct for family-wise error when appropriate
9. **Ignoring missing data**: Understand mechanism (MCAR, MAR, MNAR)
10. **Overinterpreting non-significant results**: Absence of evidence ≠ evidence of absence

---

## Getting Started Checklist

When beginning a statistical analysis:

- [ ] Define research question and hypotheses
- [ ] Determine appropriate statistical test (use test_selection_guide.md)
- [ ] Conduct power analysis to determine sample size
- [ ] Load and inspect data
- [ ] Check for missing data and outliers
- [ ] Verify assumptions using assumption_checks.py
- [ ] Run primary analysis
- [ ] Calculate effect sizes with confidence intervals
- [ ] Conduct post-hoc tests if needed (with corrections)
- [ ] Create visualizations
- [ ] Write results following reporting_standards.md
- [ ] Conduct sensitivity analyses
- [ ] Share data and code

---

## Support and Further Reading

For questions about:
- **Test selection**: See references/test_selection_guide.md
- **Assumptions**: See references/assumptions_and_diagnostics.md
- **Effect sizes**: See references/effect_sizes_and_power.md
- **Bayesian methods**: See references/bayesian_statistics.md
- **Reporting**: See references/reporting_standards.md

**Key textbooks**:
- Cohen, J. (1988). *Statistical Power Analysis for the Behavioral Sciences*
- Field, A. (2013). *Discovering Statistics Using IBM SPSS Statistics*
- Gelman, A., & Hill, J. (2006). *Data Analysis Using Regression and Multilevel/Hierarchical Models*
- Kruschke, J. K. (2014). *Doing Bayesian Data Analysis*

**Online resources**:
- APA Style Guide: https://apastyle.apa.org/
- Statistical Consulting: Cross Validated (stats.stackexchange.com)

{
  "format": "aiskill/v1",
  "skill": {
    "name": "statistical-analysis",
    "slug": "statistical-analysis",
    "version": "1.0.0",
    "description": "Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.",
    "category": "Research",
    "tags": [
      "research-science",
      "research"
    ],
    "platforms": [
      "chatgpt",
      "claude-code",
      "cursor"
    ],
    "tested_environment": "Agent Skills convention (SKILL.md) — works with Claude Code, Cursor, ChatGPT/Codex and compatible agents",
    "license": "MIT LICENSE"
  },
  "prompts": {
    "system": "# Statistical Analysis\r\n\r\n## Overview\r\n\r\nStatistical analysis is a systematic process for testing hypotheses and quantifying relationships. Conduct hypothesis tests (t-test, ANOVA, chi-square), regression, correlation, and Bayesian analyses with assumption checks and APA reporting. Apply this skill for academic research.\r\n\r\n## When to Use This Skill\r\n\r\nThis skill should be used when:\r\n- Conducting statistical hypothesis tests (t-tests, ANOVA, chi-square)\r\n- Performing regression or correlation analyses\r\n- Running Bayesian statistical analyses\r\n- Checking statistical assumptions and diagnostics\r\n- Calculating effect sizes and conducting power analyses\r\n- Reporting statistical results in APA format\r\n- Analyzing experimental or observational data for research\r\n\r\n---\r\n\r\n## Core Capabilities\r\n\r\n### 1. Test Selection and Planning\r\n- Choose appropriate statistical tests based on research questions and data characteristics\r\n- Conduct a priori power analyses to determine required sample sizes\r\n- Plan analysis strategies including multiple comparison corrections\r\n\r\n### 2. Assumption Checking\r\n- Automatically verify all relevant assumptions before running tests\r\n- Provide diagnostic visualizations (Q-Q plots, residual plots, box plots)\r\n- Recommend remedial actions when assumptions are violated\r\n\r\n### 3. Statistical Testing\r\n- Hypothesis testing: t-tests, ANOVA, chi-square, non-parametric alternatives\r\n- Regression: linear, multiple, logistic, with diagnostics\r\n- Correlations: Pearson, Spearman, with confidence intervals\r\n- Bayesian alternatives: Bayesian t-tests, ANOVA, regression with Bayes Factors\r\n\r\n### 4. Effect Sizes and Interpretation\r\n- Calculate and interpret appropriate effect sizes for all analyses\r\n- Provide confidence intervals for effect estimates\r\n- Distinguish statistical from practical significance\r\n\r\n### 5. Professional Reporting\r\n- Generate APA-style statistical reports\r\n- Create publication-ready figures and tables\r\n- Provide complete interpretation with all required statistics\r\n\r\n---\r\n\r\n## Workflow Decision Tree\r\n\r\nUse this decision tree to determine your analysis path:\r\n\r\n```\r\nSTART\r\n│\r\n├─ Need to SELECT a statistical test?\r\n│  └─ YES → See \"Test Selection Guide\"\r\n│  └─ NO → Continue\r\n│\r\n├─ Ready to check ASSUMPTIONS?\r\n│  └─ YES → See \"Assumption Checking\"\r\n│  └─ NO → Continue\r\n│\r\n├─ Ready to run ANALYSIS?\r\n│  └─ YES → See \"Running Statistical Tests\"\r\n│  └─ NO → Continue\r\n│\r\n└─ Need to REPORT results?\r\n   └─ YES → See \"Reporting Results\"\r\n```\r\n\r\n---\r\n\r\n## Test Selection Guide\r\n\r\n### Quick Reference: Choosing the Right Test\r\n\r\nUse `references/test_selection_guide.md` for comprehensive guidance. Quick reference:\r\n\r\n**Comparing Two Groups:**\r\n- Independent, continuous, normal → Independent t-test\r\n- Independent, continuous, non-normal → Mann-Whitney U test\r\n- Paired, continuous, normal → Paired t-test\r\n- Paired, continuous, non-normal → Wilcoxon signed-rank test\r\n- Binary outcome → Chi-square or Fisher's exact test\r\n\r\n**Comparing 3+ Groups:**\r\n- Independent, continuous, normal → One-way ANOVA\r\n- Independent, continuous, non-normal → Kruskal-Wallis test\r\n- Paired, continuous, normal → Repeated measures ANOVA\r\n- Paired, continuous, non-normal → Friedman test\r\n\r\n**Relationships:**\r\n- Two continuous variables → Pearson (normal) or Spearman correlation (non-normal)\r\n- Continuous outcome with predictor(s) → Linear regression\r\n- Binary outcome with predictor(s) → Logistic regression\r\n\r\n**Bayesian Alternatives:**\r\nAll tests have Bayesian versions that provide:\r\n- Direct probability statements about hypotheses\r\n- Bayes Factors quantifying evidence\r\n- Ability to support null hypothesis\r\n- See `references/bayesian_statistics.md`\r\n\r\n---\r\n\r\n## Assumption Checking\r\n\r\n### Systematic Assumption Verification\r\n\r\n**ALWAYS check assumptions before interpreting test results.**\r\n\r\nUse the provided `scripts/assumption_checks.py` module for automated checking:\r\n\r\n```python\r\nfrom scripts.assumption_checks import comprehensive_assumption_check\r\n\r\n# Comprehensive check with visualizations\r\nresults = comprehensive_assumption_check(\r\n    data=df,\r\n    value_col='score',\r\n    group_col='group',  # Optional: for group comparisons\r\n    alpha=0.05\r\n)\r\n```\r\n\r\nThis performs:\r\n1. **Outlier detection** (IQR and z-score methods)\r\n2. **Normality testing** (Shapiro-Wilk test + Q-Q plots)\r\n3. **Homogeneity of variance** (Levene's test + box plots)\r\n4. **Interpretation and recommendations**\r\n\r\n### Individual Assumption Checks\r\n\r\nFor targeted checks, use individual functions:\r\n\r\n```python\r\nfrom scripts.assumption_checks import (\r\n    check_normality,\r\n    check_normality_per_group,\r\n    check_homogeneity_of_variance,\r\n    check_linearity,\r\n    detect_outliers\r\n)\r\n\r\n# Example: Check normality with visualization\r\nresult = check_normality(\r\n    data=df['score'],\r\n    name='Test Score',\r\n    alpha=0.05,\r\n    plot=True\r\n)\r\nprint(result['interpretation'])\r\nprint(result['recommendation'])\r\n```\r\n\r\n### What to Do When Assumptions Are Violated\r\n\r\n**Normality violated:**\r\n- Mild violation + n > 30 per group → Proceed with parametric test (robust)\r\n- Moderate violation → Use non-parametric alternative\r\n- Severe violation → Transform data or use non-parametric test\r\n\r\n**Homogeneity of variance violated:**\r\n- For t-test → Use Welch's t-test\r\n- For ANOVA → Use Welch's ANOVA or Brown-Forsythe ANOVA\r\n- For regression → Use robust standard errors or weighted least squares\r\n\r\n**Linearity violated (regression):**\r\n- Add polynomial terms\r\n- Transform variables\r\n- Use non-linear models or GAM\r\n\r\nSee `references/assumptions_and_diagnostics.md` for comprehensive guidance.\r\n\r\n---\r\n\r\n## Running Statistical Tests\r\n\r\n### Python Libraries\r\n\r\nPrimary libraries for statistical analysis:\r\n- **scipy.stats**: Core statistical tests\r\n- **statsmodels**: Advanced regression and diagnostics\r\n- **pingouin**: User-friendly statistical testing with effect sizes\r\n- **pymc**: Bayesian statistical modeling\r\n- **arviz**: Bayesian visualization and diagnostics\r\n\r\n### Example Analyses\r\n\r\n#### T-Test with Complete Reporting\r\n\r\n```python\r\nimport pingouin as pg\r\nimport numpy as np\r\n\r\n# Run independent t-test\r\nresult = pg.ttest(group_a, group_b, correction='auto')\r\n\r\n# Extract results\r\nt_stat = result['T'].values[0]\r\ndf = result['dof'].values[0]\r\np_value = result['p-val'].values[0]\r\ncohens_d = result['cohen-d'].values[0]\r\nci_lower = result['CI95%'].values[0][0]\r\nci_upper = result['CI95%'].values[0][1]\r\n\r\n# Report\r\nprint(f\"t({df:.0f}) = {t_stat:.2f}, p = {p_value:.3f}\")\r\nprint(f\"Cohen's d = {cohens_d:.2f}, 95% CI [{ci_lower:.2f}, {ci_upper:.2f}]\")\r\n```\r\n\r\n#### ANOVA with Post-Hoc Tests\r\n\r\n```python\r\nimport pingouin as pg\r\n\r\n# One-way ANOVA\r\naov = pg.anova(dv='score', between='group', data=df, detailed=True)\r\nprint(aov)\r\n\r\n# If significant, conduct post-hoc tests\r\nif aov['p-unc'].values[0] < 0.05:\r\n    posthoc = pg.pairwise_tukey(dv='score', between='group', data=df)\r\n    print(posthoc)\r\n\r\n# Effect size\r\neta_squared = aov['np2'].values[0]  # Partial eta-squared\r\nprint(f\"Partial η² = {eta_squared:.3f}\")\r\n```\r\n\r\n#### Linear Regression with Diagnostics\r\n\r\n```python\r\nimport statsmodels.api as sm\r\nfrom statsmodels.stats.outliers_influence import variance_inflation_factor\r\n\r\n# Fit model\r\nX = sm.add_constant(X_predictors)  # Add intercept\r\nmodel = sm.OLS(y, X).fit()\r\n\r\n# Summary\r\nprint(model.summary())\r\n\r\n# Check multicollinearity (VIF)\r\nvif_data = pd.DataFrame()\r\nvif_data[\"Variable\"] = X.columns\r\nvif_data[\"VIF\"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]\r\nprint(vif_data)\r\n\r\n# Check assumptions\r\nresiduals = model.resid\r\nfitted = model.fittedvalues\r\n\r\n# Residual plots\r\nimport matplotlib.pyplot as plt\r\nfig, axes = plt.subplots(2, 2, figsize=(12, 10))\r\n\r\n# Residuals vs fitted\r\naxes[0, 0].scatter(fitted, residuals, alpha=0.6)\r\naxes[0, 0].axhline(y=0, color='r', linestyle='--')\r\naxes[0, 0].set_xlabel('Fitted values')\r\naxes[0, 0].set_ylabel('Residuals')\r\naxes[0, 0].set_title('Residuals vs Fitted')\r\n\r\n# Q-Q plot\r\nfrom scipy import stats\r\nstats.probplot(residuals, dist=\"norm\", plot=axes[0, 1])\r\naxes[0, 1].set_title('Normal Q-Q')\r\n\r\n# Scale-Location\r\naxes[1, 0].scatter(fitted, np.sqrt(np.abs(residuals / residuals.std())), alpha=0.6)\r\naxes[1, 0].set_xlabel('Fitted values')\r\naxes[1, 0].set_ylabel('√|Standardized residuals|')\r\naxes[1, 0].set_title('Scale-Location')\r\n\r\n# Residuals histogram\r\naxes[1, 1].hist(residuals, bins=20, edgecolor='black', alpha=0.7)\r\naxes[1, 1].set_xlabel('Residuals')\r\naxes[1, 1].set_ylabel('Frequency')\r\naxes[1, 1].set_title('Histogram of Residuals')\r\n\r\nplt.tight_layout()\r\nplt.show()\r\n```\r\n\r\n#### Bayesian T-Test\r\n\r\n```python\r\nimport pymc as pm\r\nimport arviz as az\r\nimport numpy as np\r\n\r\nwith pm.Model() as model:\r\n    # Priors\r\n    mu1 = pm.Normal('mu_group1', mu=0, sigma=10)\r\n    mu2 = pm.Normal('mu_group2', mu=0, sigma=10)\r\n    sigma = pm.HalfNormal('sigma', sigma=10)\r\n\r\n    # Likelihood\r\n    y1 = pm.Normal('y1', mu=mu1, sigma=sigma, observed=group_a)\r\n    y2 = pm.Normal('y2', mu=mu2, sigma=sigma, observed=group_b)\r\n\r\n    # Derived quantity\r\n    diff = pm.Deterministic('difference', mu1 - mu2)\r\n\r\n    # Sample\r\n    trace = pm.sample(2000, tune=1000, return_inferencedata=True)\r\n\r\n# Summarize\r\nprint(az.summary(trace, var_names=['difference']))\r\n\r\n# Probability that group1 > group2\r\nprob_greater = np.mean(trace.posterior['difference'].values > 0)\r\nprint(f\"P(μ₁ > μ₂ | data) = {prob_greater:.3f}\")\r\n\r\n# Plot posterior\r\naz.plot_posterior(trace, var_names=['difference'], ref_val=0)\r\n```\r\n\r\n---\r\n\r\n## Effect Sizes\r\n\r\n### Always Calculate Effect Sizes\r\n\r\n**Effect sizes quantify magnitude, while p-values only indicate existence of an effect.**\r\n\r\nSee `references/effect_sizes_and_power.md` for comprehensive guidance.\r\n\r\n### Quick Reference: Common Effect Sizes\r\n\r\n| Test | Effect Size | Small | Medium | Large |\r\n|------|-------------|-------|--------|-------|\r\n| T-test | Cohen's d | 0.20 | 0.50 | 0.80 |\r\n| ANOVA | η²_p | 0.01 | 0.06 | 0.14 |\r\n| Correlation | r | 0.10 | 0.30 | 0.50 |\r\n| Regression | R² | 0.02 | 0.13 | 0.26 |\r\n| Chi-square | Cramér's V | 0.07 | 0.21 | 0.35 |\r\n\r\n**Important**: Benchmarks are guidelines. Context matters!\r\n\r\n### Calculating Effect Sizes\r\n\r\nMost effect sizes are automatically calculated by pingouin:\r\n\r\n```python\r\n# T-test returns Cohen's d\r\nresult = pg.ttest(x, y)\r\nd = result['cohen-d'].values[0]\r\n\r\n# ANOVA returns partial eta-squared\r\naov = pg.anova(dv='score', between='group', data=df)\r\neta_p2 = aov['np2'].values[0]\r\n\r\n# Correlation: r is already an effect size\r\ncorr = pg.corr(x, y)\r\nr = corr['r'].values[0]\r\n```\r\n\r\n### Confidence Intervals for Effect Sizes\r\n\r\nAlways report CIs to show precision:\r\n\r\n```python\r\nfrom pingouin import compute_effsize_from_t\r\n\r\n# For t-test\r\nd, ci = compute_effsize_from_t(\r\n    t_statistic,\r\n    nx=len(group1),\r\n    ny=len(group2),\r\n    eftype='cohen'\r\n)\r\nprint(f\"d = {d:.2f}, 95% CI [{ci[0]:.2f}, {ci[1]:.2f}]\")\r\n```\r\n\r\n---\r\n\r\n## Power Analysis\r\n\r\n### A Priori Power Analysis (Study Planning)\r\n\r\nDetermine required sample size before data collection:\r\n\r\n```python\r\nfrom statsmodels.stats.power import (\r\n    tt_ind_solve_power,\r\n    FTestAnovaPower\r\n)\r\n\r\n# T-test: What n is needed to detect d = 0.5?\r\nn_required = tt_ind_solve_power(\r\n    effect_size=0.5,\r\n    alpha=0.05,\r\n    power=0.80,\r\n    ratio=1.0,\r\n    alternative='two-sided'\r\n)\r\nprint(f\"Required n per group: {n_required:.0f}\")\r\n\r\n# ANOVA: What n is needed to detect f = 0.25?\r\nanova_power = FTestAnovaPower()\r\nn_per_group = anova_power.solve_power(\r\n    effect_size=0.25,\r\n    ngroups=3,\r\n    alpha=0.05,\r\n    power=0.80\r\n)\r\nprint(f\"Required n per group: {n_per_group:.0f}\")\r\n```\r\n\r\n### Sensitivity Analysis (Post-Study)\r\n\r\nDetermine what effect size you could detect:\r\n\r\n```python\r\n# With n=50 per group, what effect could we detect?\r\ndetectable_d = tt_ind_solve_power(\r\n    effect_size=None,  # Solve for this\r\n    nobs1=50,\r\n    alpha=0.05,\r\n    power=0.80,\r\n    ratio=1.0,\r\n    alternative='two-sided'\r\n)\r\nprint(f\"Study could detect d ≥ {detectable_d:.2f}\")\r\n```\r\n\r\n**Note**: Post-hoc power analysis (calculating power after study) is generally not recommended. Use sensitivity analysis instead.\r\n\r\nSee `references/effect_sizes_and_power.md` for detailed guidance.\r\n\r\n---\r\n\r\n## Reporting Results\r\n\r\n### APA Style Statistical Reporting\r\n\r\nFollow guidelines in `references/reporting_standards.md`.\r\n\r\n### Essential Reporting Elements\r\n\r\n1. **Descriptive statistics**: M, SD, n for all groups/variables\r\n2. **Test statistics**: Test name, statistic, df, exact p-value\r\n3. **Effect sizes**: With confidence intervals\r\n4. **Assumption checks**: Which tests were done, results, actions taken\r\n5. **All planned analyses**: Including non-significant findings\r\n\r\n### Example Report Templates\r\n\r\n#### Independent T-Test\r\n\r\n```\r\nGroup A (n = 48, M = 75.2, SD = 8.5) scored significantly higher than\r\nGroup B (n = 52, M = 68.3, SD = 9.2), t(98) = 3.82, p < .001, d = 0.77,\r\n95% CI [0.36, 1.18], two-tailed. Assumptions of normality (Shapiro-Wilk:\r\nGroup A W = 0.97, p = .18; Group B W = 0.96, p = .12) and homogeneity\r\nof variance (Levene's F(1, 98) = 1.23, p = .27) were satisfied.\r\n```\r\n\r\n#### One-Way ANOVA\r\n\r\n```\r\nA one-way ANOVA revealed a significant main effect of treatment condition\r\non test scores, F(2, 147) = 8.45, p < .001, η²_p = .10. Post hoc\r\ncomparisons using Tukey's HSD indicated that Condition A (M = 78.2,\r\nSD = 7.3) scored significantly higher than Condition B (M = 71.5,\r\nSD = 8.1, p = .002, d = 0.87) and Condition C (M = 70.1, SD = 7.9,\r\np < .001, d = 1.07). Conditions B and C did not differ significantly\r\n(p = .52, d = 0.18).\r\n```\r\n\r\n#### Multiple Regression\r\n\r\n```\r\nMultiple linear regression was conducted to predict exam scores from\r\nstudy hours, prior GPA, and attendance. The overall model was significant,\r\nF(3, 146) = 45.2, p < .001, R² = .48, adjusted R² = .47. Study hours\r\n(B = 1.80, SE = 0.31, β = .35, t = 5.78, p < .001, 95% CI [1.18, 2.42])\r\nand prior GPA (B = 8.52, SE = 1.95, β = .28, t = 4.37, p < .001,\r\n95% CI [4.66, 12.38]) were significant predictors, while attendance was\r\nnot (B = 0.15, SE = 0.12, β = .08, t = 1.25, p = .21, 95% CI [-0.09, 0.39]).\r\nMulticollinearity was not a concern (all VIF < 1.5).\r\n```\r\n\r\n#### Bayesian Analysis\r\n\r\n```\r\nA Bayesian independent samples t-test was conducted using weakly\r\ninformative priors (Normal(0, 1) for mean difference). The posterior\r\ndistribution indicated that Group A scored higher than Group B\r\n(M_diff = 6.8, 95% credible interval [3.2, 10.4]). The Bayes Factor\r\nBF₁₀ = 45.3 provided very strong evidence for a difference between\r\ngroups, with a 99.8% posterior probability that Group A's mean exceeded\r\nGroup B's mean. Convergence diagnostics were satisfactory (all R̂ < 1.01,\r\nESS > 1000).\r\n```\r\n\r\n---\r\n\r\n## Bayesian Statistics\r\n\r\n### When to Use Bayesian Methods\r\n\r\nConsider Bayesian approaches when:\r\n- You have prior information to incorporate\r\n- You want direct probability statements about hypotheses\r\n- Sample size is small or planning sequential data collection\r\n- You need to quantify evidence for the null hypothesis\r\n- The model is complex (hierarchical, missing data)\r\n\r\nSee `references/bayesian_statistics.md` for comprehensive guidance on:\r\n- Bayes' theorem and interpretation\r\n- Prior specification (informative, weakly informative, non-informative)\r\n- Bayesian hypothesis testing with Bayes Factors\r\n- Credible intervals vs. confidence intervals\r\n- Bayesian t-tests, ANOVA, regression, and hierarchical models\r\n- Model convergence checking and posterior predictive checks\r\n\r\n### Key Advantages\r\n\r\n1. **Intuitive interpretation**: \"Given the data, there is a 95% probability the parameter is in this interval\"\r\n2. **Evidence for null**: Can quantify support for no effect\r\n3. **Flexible**: No p-hacking concerns; can analyze data as it arrives\r\n4. **Uncertainty quantification**: Full posterior distribution\r\n\r\n---\r\n\r\n## Resources\r\n\r\nThis skill includes comprehensive reference materials:\r\n\r\n### References Directory\r\n\r\n- **test_selection_guide.md**: Decision tree for choosing appropriate statistical tests\r\n- **assumptions_and_diagnostics.md**: Detailed guidance on checking and handling assumption violations\r\n- **effect_sizes_and_power.md**: Calculating, interpreting, and reporting effect sizes; conducting power analyses\r\n- **bayesian_statistics.md**: Complete guide to Bayesian analysis methods\r\n- **reporting_standards.md**: APA-style reporting guidelines with examples\r\n\r\n### Scripts Directory\r\n\r\n- **assumption_checks.py**: Automated assumption checking with visualizations\r\n  - `comprehensive_assumption_check()`: Complete workflow\r\n  - `check_normality()`: Normality testing with Q-Q plots\r\n  - `check_homogeneity_of_variance()`: Levene's test with box plots\r\n  - `check_linearity()`: Regression linearity checks\r\n  - `detect_outliers()`: IQR and z-score outlier detection\r\n\r\n---\r\n\r\n## Best Practices\r\n\r\n1. **Pre-register analyses** when possible to distinguish confirmatory from exploratory\r\n2. **Always check assumptions** before interpreting results\r\n3. **Report effect sizes** with confidence intervals\r\n4. **Report all planned analyses** including non-significant results\r\n5. **Distinguish statistical from practical significance**\r\n6. **Visualize data** before and after analysis\r\n7. **Check diagnostics** for regression/ANOVA (residual plots, VIF, etc.)\r\n8. **Conduct sensitivity analyses** to assess robustness\r\n9. **Share data and code** for reproducibility\r\n10. **Be transparent** about violations, transformations, and decisions\r\n\r\n---\r\n\r\n## Common Pitfalls to Avoid\r\n\r\n1. **P-hacking**: Don't test multiple ways until something is significant\r\n2. **HARKing**: Don't present exploratory findings as confirmatory\r\n3. **Ignoring assumptions**: Check them and report violations\r\n4. **Confusing significance with importance**: p < .05 ≠ meaningful effect\r\n5. **Not reporting effect sizes**: Essential for interpretation\r\n6. **Cherry-picking results**: Report all planned analyses\r\n7. **Misinterpreting p-values**: They're NOT probability that hypothesis is true\r\n8. **Multiple comparisons**: Correct for family-wise error when appropriate\r\n9. **Ignoring missing data**: Understand mechanism (MCAR, MAR, MNAR)\r\n10. **Overinterpreting non-significant results**: Absence of evidence ≠ evidence of absence\r\n\r\n---\r\n\r\n## Getting Started Checklist\r\n\r\nWhen beginning a statistical analysis:\r\n\r\n- [ ] Define research question and hypotheses\r\n- [ ] Determine appropriate statistical test (use test_selection_guide.md)\r\n- [ ] Conduct power analysis to determine sample size\r\n- [ ] Load and inspect data\r\n- [ ] Check for missing data and outliers\r\n- [ ] Verify assumptions using assumption_checks.py\r\n- [ ] Run primary analysis\r\n- [ ] Calculate effect sizes with confidence intervals\r\n- [ ] Conduct post-hoc tests if needed (with corrections)\r\n- [ ] Create visualizations\r\n- [ ] Write results following reporting_standards.md\r\n- [ ] Conduct sensitivity analyses\r\n- [ ] Share data and code\r\n\r\n---\r\n\r\n## Support and Further Reading\r\n\r\nFor questions about:\r\n- **Test selection**: See references/test_selection_guide.md\r\n- **Assumptions**: See references/assumptions_and_diagnostics.md\r\n- **Effect sizes**: See references/effect_sizes_and_power.md\r\n- **Bayesian methods**: See references/bayesian_statistics.md\r\n- **Reporting**: See references/reporting_standards.md\r\n\r\n**Key textbooks**:\r\n- Cohen, J. (1988). *Statistical Power Analysis for the Behavioral Sciences*\r\n- Field, A. (2013). *Discovering Statistics Using IBM SPSS Statistics*\r\n- Gelman, A., & Hill, J. (2006). *Data Analysis Using Regression and Multilevel/Hierarchical Models*\r\n- Kruschke, J. K. (2014). *Doing Bayesian Data Analysis*\r\n\r\n**Online resources**:\r\n- APA Style Guide: https://apastyle.apa.org/\r\n- Statistical Consulting: Cross Validated (stats.stackexchange.com)",
    "user_template": ""
  },
  "variables": [],
  "install": {
    "steps": [
      {
        "title": "Download & unzip",
        "detail": "Download the .zip package from SkillVault and unzip it. The statistical-analysis/ folder contains SKILL.md plus any companion scripts, references and assets."
      },
      {
        "title": "Copy into your agent skills directory",
        "detail": "Move the folder into your agent's skills directory — e.g. ~/.claude/skills/ (Claude Code), .codex/skills/, .cursor/skills/ or your project's .claude/skills/ folder — then restart or reload the session."
      },
      {
        "title": "Use it",
        "detail": "Describe the task in natural language; the agent loads the skill automatically when the request matches the skill description."
      }
    ]
  },
  "usage": {
    "steps": []
  },
  "metadata": {
    "author": "SkillVault",
    "source": "https://skillvault.top/skills/statistical-analysis",
    "exported_at": "2026-11-03T07:42:11.579Z"
  }
}

format: aiskill/v1
skill:
  name: statistical-analysis
  slug: statistical-analysis
  version: 1.0.0
  description: >-
    Guided statistical analysis with test selection and reporting. Use when you need help choosing
    appropriate tests for your data, assumption checking, power analysis, and APA-formatted results.
    Best for academic research reporting, test selection guidance. For implementing specific models
    programmatically use statsmodels.
  category: Research
  tags:
    - research-science
    - research
  platforms:
    - chatgpt
    - claude-code
    - cursor
  tested_environment: >-
    Agent Skills convention (SKILL.md) — works with Claude Code, Cursor, ChatGPT/Codex and
    compatible agents
  license: MIT LICENSE
prompts:
  system: "# Statistical Analysis\r\n\r\n## Overview\r\n\r\nStatistical analysis is a systematic process for testing hypotheses and quantifying relationships. Conduct hypothesis tests (t-test, ANOVA, chi-square), regression, correlation, and Bayesian analyses with assumption checks and APA reporting. Apply this skill for academic research.\r\n\r\n## When to Use This Skill\r\n\r\nThis skill should be used when:\r\n- Conducting statistical hypothesis tests (t-tests, ANOVA, chi-square)\r\n- Performing regression or correlation analyses\r\n- Running Bayesian statistical analyses\r\n- Checking statistical assumptions and diagnostics\r\n- Calculating effect sizes and conducting power analyses\r\n- Reporting statistical results in APA format\r\n- Analyzing experimental or observational data for research\r\n\r\n---\r\n\r\n## Core Capabilities\r\n\r\n### 1. Test Selection and Planning\r\n- Choose appropriate statistical tests based on research questions and data characteristics\r\n- Conduct a priori power analyses to determine required sample sizes\r\n- Plan analysis strategies including multiple comparison corrections\r\n\r\n### 2. Assumption Checking\r\n- Automatically verify all relevant assumptions before running tests\r\n- Provide diagnostic visualizations (Q-Q plots, residual plots, box plots)\r\n- Recommend remedial actions when assumptions are violated\r\n\r\n### 3. Statistical Testing\r\n- Hypothesis testing: t-tests, ANOVA, chi-square, non-parametric alternatives\r\n- Regression: linear, multiple, logistic, with diagnostics\r\n- Correlations: Pearson, Spearman, with confidence intervals\r\n- Bayesian alternatives: Bayesian t-tests, ANOVA, regression with Bayes Factors\r\n\r\n### 4. Effect Sizes and Interpretation\r\n- Calculate and interpret appropriate effect sizes for all analyses\r\n- Provide confidence intervals for effect estimates\r\n- Distinguish statistical from practical significance\r\n\r\n### 5. Professional Reporting\r\n- Generate APA-style statistical reports\r\n- Create publication-ready figures and tables\r\n- Provide complete interpretation with all required statistics\r\n\r\n---\r\n\r\n## Workflow Decision Tree\r\n\r\nUse this decision tree to determine your analysis path:\r\n\r\n```\r\nSTART\r\n│\r\n├─ Need to SELECT a statistical test?\r\n│  └─ YES → See \"Test Selection Guide\"\r\n│  └─ NO → Continue\r\n│\r\n├─ Ready to check ASSUMPTIONS?\r\n│  └─ YES → See \"Assumption Checking\"\r\n│  └─ NO → Continue\r\n│\r\n├─ Ready to run ANALYSIS?\r\n│  └─ YES → See \"Running Statistical Tests\"\r\n│  └─ NO → Continue\r\n│\r\n└─ Need to REPORT results?\r\n   └─ YES → See \"Reporting Results\"\r\n```\r\n\r\n---\r\n\r\n## Test Selection Guide\r\n\r\n### Quick Reference: Choosing the Right Test\r\n\r\nUse `references/test_selection_guide.md` for comprehensive guidance. Quick reference:\r\n\r\n**Comparing Two Groups:**\r\n- Independent, continuous, normal → Independent t-test\r\n- Independent, continuous, non-normal → Mann-Whitney U test\r\n- Paired, continuous, normal → Paired t-test\r\n- Paired, continuous, non-normal → Wilcoxon signed-rank test\r\n- Binary outcome → Chi-square or Fisher's exact test\r\n\r\n**Comparing 3+ Groups:**\r\n- Independent, continuous, normal → One-way ANOVA\r\n- Independent, continuous, non-normal → Kruskal-Wallis test\r\n- Paired, continuous, normal → Repeated measures ANOVA\r\n- Paired, continuous, non-normal → Friedman test\r\n\r\n**Relationships:**\r\n- Two continuous variables → Pearson (normal) or Spearman correlation (non-normal)\r\n- Continuous outcome with predictor(s) → Linear regression\r\n- Binary outcome with predictor(s) → Logistic regression\r\n\r\n**Bayesian Alternatives:**\r\nAll tests have Bayesian versions that provide:\r\n- Direct probability statements about hypotheses\r\n- Bayes Factors quantifying evidence\r\n- Ability to support null hypothesis\r\n- See `references/bayesian_statistics.md`\r\n\r\n---\r\n\r\n## Assumption Checking\r\n\r\n### Systematic Assumption Verification\r\n\r\n**ALWAYS check assumptions before interpreting test results.**\r\n\r\nUse the provided `scripts/assumption_checks.py` module for automated checking:\r\n\r\n```python\r\nfrom scripts.assumption_checks import comprehensive_assumption_check\r\n\r\n# Comprehensive check with visualizations\r\nresults = comprehensive_assumption_check(\r\n    data=df,\r\n    value_col='score',\r\n    group_col='group',  # Optional: for group comparisons\r\n    alpha=0.05\r\n)\r\n```\r\n\r\nThis performs:\r\n1. **Outlier detection** (IQR and z-score methods)\r\n2. **Normality testing** (Shapiro-Wilk test + Q-Q plots)\r\n3. **Homogeneity of variance** (Levene's test + box plots)\r\n4. **Interpretation and recommendations**\r\n\r\n### Individual Assumption Checks\r\n\r\nFor targeted checks, use individual functions:\r\n\r\n```python\r\nfrom scripts.assumption_checks import (\r\n    check_normality,\r\n    check_normality_per_group,\r\n    check_homogeneity_of_variance,\r\n    check_linearity,\r\n    detect_outliers\r\n)\r\n\r\n# Example: Check normality with visualization\r\nresult = check_normality(\r\n    data=df['score'],\r\n    name='Test Score',\r\n    alpha=0.05,\r\n    plot=True\r\n)\r\nprint(result['interpretation'])\r\nprint(result['recommendation'])\r\n```\r\n\r\n### What to Do When Assumptions Are Violated\r\n\r\n**Normality violated:**\r\n- Mild violation + n > 30 per group → Proceed with parametric test (robust)\r\n- Moderate violation → Use non-parametric alternative\r\n- Severe violation → Transform data or use non-parametric test\r\n\r\n**Homogeneity of variance violated:**\r\n- For t-test → Use Welch's t-test\r\n- For ANOVA → Use Welch's ANOVA or Brown-Forsythe ANOVA\r\n- For regression → Use robust standard errors or weighted least squares\r\n\r\n**Linearity violated (regression):**\r\n- Add polynomial terms\r\n- Transform variables\r\n- Use non-linear models or GAM\r\n\r\nSee `references/assumptions_and_diagnostics.md` for comprehensive guidance.\r\n\r\n---\r\n\r\n## Running Statistical Tests\r\n\r\n### Python Libraries\r\n\r\nPrimary libraries for statistical analysis:\r\n- **scipy.stats**: Core statistical tests\r\n- **statsmodels**: Advanced regression and diagnostics\r\n- **pingouin**: User-friendly statistical testing with effect sizes\r\n- **pymc**: Bayesian statistical modeling\r\n- **arviz**: Bayesian visualization and diagnostics\r\n\r\n### Example Analyses\r\n\r\n#### T-Test with Complete Reporting\r\n\r\n```python\r\nimport pingouin as pg\r\nimport numpy as np\r\n\r\n# Run independent t-test\r\nresult = pg.ttest(group_a, group_b, correction='auto')\r\n\r\n# Extract results\r\nt_stat = result['T'].values[0]\r\ndf = result['dof'].values[0]\r\np_value = result['p-val'].values[0]\r\ncohens_d = result['cohen-d'].values[0]\r\nci_lower = result['CI95%'].values[0][0]\r\nci_upper = result['CI95%'].values[0][1]\r\n\r\n# Report\r\nprint(f\"t({df:.0f}) = {t_stat:.2f}, p = {p_value:.3f}\")\r\nprint(f\"Cohen's d = {cohens_d:.2f}, 95% CI [{ci_lower:.2f}, {ci_upper:.2f}]\")\r\n```\r\n\r\n#### ANOVA with Post-Hoc Tests\r\n\r\n```python\r\nimport pingouin as pg\r\n\r\n# One-way ANOVA\r\naov = pg.anova(dv='score', between='group', data=df, detailed=True)\r\nprint(aov)\r\n\r\n# If significant, conduct post-hoc tests\r\nif aov['p-unc'].values[0] < 0.05:\r\n    posthoc = pg.pairwise_tukey(dv='score', between='group', data=df)\r\n    print(posthoc)\r\n\r\n# Effect size\r\neta_squared = aov['np2'].values[0]  # Partial eta-squared\r\nprint(f\"Partial η² = {eta_squared:.3f}\")\r\n```\r\n\r\n#### Linear Regression with Diagnostics\r\n\r\n```python\r\nimport statsmodels.api as sm\r\nfrom statsmodels.stats.outliers_influence import variance_inflation_factor\r\n\r\n# Fit model\r\nX = sm.add_constant(X_predictors)  # Add intercept\r\nmodel = sm.OLS(y, X).fit()\r\n\r\n# Summary\r\nprint(model.summary())\r\n\r\n# Check multicollinearity (VIF)\r\nvif_data = pd.DataFrame()\r\nvif_data[\"Variable\"] = X.columns\r\nvif_data[\"VIF\"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]\r\nprint(vif_data)\r\n\r\n# Check assumptions\r\nresiduals = model.resid\r\nfitted = model.fittedvalues\r\n\r\n# Residual plots\r\nimport matplotlib.pyplot as plt\r\nfig, axes = plt.subplots(2, 2, figsize=(12, 10))\r\n\r\n# Residuals vs fitted\r\naxes[0, 0].scatter(fitted, residuals, alpha=0.6)\r\naxes[0, 0].axhline(y=0, color='r', linestyle='--')\r\naxes[0, 0].set_xlabel('Fitted values')\r\naxes[0, 0].set_ylabel('Residuals')\r\naxes[0, 0].set_title('Residuals vs Fitted')\r\n\r\n# Q-Q plot\r\nfrom scipy import stats\r\nstats.probplot(residuals, dist=\"norm\", plot=axes[0, 1])\r\naxes[0, 1].set_title('Normal Q-Q')\r\n\r\n# Scale-Location\r\naxes[1, 0].scatter(fitted, np.sqrt(np.abs(residuals / residuals.std())), alpha=0.6)\r\naxes[1, 0].set_xlabel('Fitted values')\r\naxes[1, 0].set_ylabel('√|Standardized residuals|')\r\naxes[1, 0].set_title('Scale-Location')\r\n\r\n# Residuals histogram\r\naxes[1, 1].hist(residuals, bins=20, edgecolor='black', alpha=0.7)\r\naxes[1, 1].set_xlabel('Residuals')\r\naxes[1, 1].set_ylabel('Frequency')\r\naxes[1, 1].set_title('Histogram of Residuals')\r\n\r\nplt.tight_layout()\r\nplt.show()\r\n```\r\n\r\n#### Bayesian T-Test\r\n\r\n```python\r\nimport pymc as pm\r\nimport arviz as az\r\nimport numpy as np\r\n\r\nwith pm.Model() as model:\r\n    # Priors\r\n    mu1 = pm.Normal('mu_group1', mu=0, sigma=10)\r\n    mu2 = pm.Normal('mu_group2', mu=0, sigma=10)\r\n    sigma = pm.HalfNormal('sigma', sigma=10)\r\n\r\n    # Likelihood\r\n    y1 = pm.Normal('y1', mu=mu1, sigma=sigma, observed=group_a)\r\n    y2 = pm.Normal('y2', mu=mu2, sigma=sigma, observed=group_b)\r\n\r\n    # Derived quantity\r\n    diff = pm.Deterministic('difference', mu1 - mu2)\r\n\r\n    # Sample\r\n    trace = pm.sample(2000, tune=1000, return_inferencedata=True)\r\n\r\n# Summarize\r\nprint(az.summary(trace, var_names=['difference']))\r\n\r\n# Probability that group1 > group2\r\nprob_greater = np.mean(trace.posterior['difference'].values > 0)\r\nprint(f\"P(μ₁ > μ₂ | data) = {prob_greater:.3f}\")\r\n\r\n# Plot posterior\r\naz.plot_posterior(trace, var_names=['difference'], ref_val=0)\r\n```\r\n\r\n---\r\n\r\n## Effect Sizes\r\n\r\n### Always Calculate Effect Sizes\r\n\r\n**Effect sizes quantify magnitude, while p-values only indicate existence of an effect.**\r\n\r\nSee `references/effect_sizes_and_power.md` for comprehensive guidance.\r\n\r\n### Quick Reference: Common Effect Sizes\r\n\r\n| Test | Effect Size | Small | Medium | Large |\r\n|------|-------------|-------|--------|-------|\r\n| T-test | Cohen's d | 0.20 | 0.50 | 0.80 |\r\n| ANOVA | η²_p | 0.01 | 0.06 | 0.14 |\r\n| Correlation | r | 0.10 | 0.30 | 0.50 |\r\n| Regression | R² | 0.02 | 0.13 | 0.26 |\r\n| Chi-square | Cramér's V | 0.07 | 0.21 | 0.35 |\r\n\r\n**Important**: Benchmarks are guidelines. Context matters!\r\n\r\n### Calculating Effect Sizes\r\n\r\nMost effect sizes are automatically calculated by pingouin:\r\n\r\n```python\r\n# T-test returns Cohen's d\r\nresult = pg.ttest(x, y)\r\nd = result['cohen-d'].values[0]\r\n\r\n# ANOVA returns partial eta-squared\r\naov = pg.anova(dv='score', between='group', data=df)\r\neta_p2 = aov['np2'].values[0]\r\n\r\n# Correlation: r is already an effect size\r\ncorr = pg.corr(x, y)\r\nr = corr['r'].values[0]\r\n```\r\n\r\n### Confidence Intervals for Effect Sizes\r\n\r\nAlways report CIs to show precision:\r\n\r\n```python\r\nfrom pingouin import compute_effsize_from_t\r\n\r\n# For t-test\r\nd, ci = compute_effsize_from_t(\r\n    t_statistic,\r\n    nx=len(group1),\r\n    ny=len(group2),\r\n    eftype='cohen'\r\n)\r\nprint(f\"d = {d:.2f}, 95% CI [{ci[0]:.2f}, {ci[1]:.2f}]\")\r\n```\r\n\r\n---\r\n\r\n## Power Analysis\r\n\r\n### A Priori Power Analysis (Study Planning)\r\n\r\nDetermine required sample size before data collection:\r\n\r\n```python\r\nfrom statsmodels.stats.power import (\r\n    tt_ind_solve_power,\r\n    FTestAnovaPower\r\n)\r\n\r\n# T-test: What n is needed to detect d = 0.5?\r\nn_required = tt_ind_solve_power(\r\n    effect_size=0.5,\r\n    alpha=0.05,\r\n    power=0.80,\r\n    ratio=1.0,\r\n    alternative='two-sided'\r\n)\r\nprint(f\"Required n per group: {n_required:.0f}\")\r\n\r\n# ANOVA: What n is needed to detect f = 0.25?\r\nanova_power = FTestAnovaPower()\r\nn_per_group = anova_power.solve_power(\r\n    effect_size=0.25,\r\n    ngroups=3,\r\n    alpha=0.05,\r\n    power=0.80\r\n)\r\nprint(f\"Required n per group: {n_per_group:.0f}\")\r\n```\r\n\r\n### Sensitivity Analysis (Post-Study)\r\n\r\nDetermine what effect size you could detect:\r\n\r\n```python\r\n# With n=50 per group, what effect could we detect?\r\ndetectable_d = tt_ind_solve_power(\r\n    effect_size=None,# Solve for this\r\n    nobs1=50,\r\n    alpha=0.05,\r\n    power=0.80,\r\n    ratio=1.0,\r\n    alternative='two-sided'\r\n)\r\nprint(f\"Study could detect d ≥ {detectable_d:.2f}\")\r\n```\r\n\r\n**Note**: Post-hoc power analysis (calculating power after study) is generally not recommended. Use sensitivity analysis instead.\r\n\r\nSee `references/effect_sizes_and_power.md` for detailed guidance.\r\n\r\n---\r\n\r\n## Reporting Results\r\n\r\n### APA Style Statistical Reporting\r\n\r\nFollow guidelines in `references/reporting_standards.md`.\r\n\r\n### Essential Reporting Elements\r\n\r\n1. **Descriptive statistics**: M, SD, n for all groups/variables\r\n2. **Test statistics**: Test name, statistic, df, exact p-value\r\n3. **Effect sizes**: With confidence intervals\r\n4. **Assumption checks**: Which tests were done, results, actions taken\r\n5. **All planned analyses**: Including non-significant findings\r\n\r\n### Example Report Templates\r\n\r\n#### Independent T-Test\r\n\r\n```\r\nGroup A (n = 48, M = 75.2, SD = 8.5) scored significantly higher than\r\nGroup B (n = 52, M = 68.3, SD = 9.2), t(98) = 3.82, p < .001, d = 0.77,\r\n95% CI [0.36, 1.18], two-tailed. Assumptions of normality (Shapiro-Wilk:\r\nGroup A W = 0.97, p = .18; Group B W = 0.96, p = .12) and homogeneity\r\nof variance (Levene's F(1, 98) = 1.23, p = .27) were satisfied.\r\n```\r\n\r\n#### One-Way ANOVA\r\n\r\n```\r\nA one-way ANOVA revealed a significant main effect of treatment condition\r\non test scores, F(2, 147) = 8.45, p < .001, η²_p = .10. Post hoc\r\ncomparisons using Tukey's HSD indicated that Condition A (M = 78.2,\r\nSD = 7.3) scored significantly higher than Condition B (M = 71.5,\r\nSD = 8.1, p = .002, d = 0.87) and Condition C (M = 70.1, SD = 7.9,\r\np < .001, d = 1.07). Conditions B and C did not differ significantly\r\n(p = .52, d = 0.18).\r\n```\r\n\r\n#### Multiple Regression\r\n\r\n```\r\nMultiple linear regression was conducted to predict exam scores from\r\nstudy hours, prior GPA, and attendance. The overall model was significant,\r\nF(3, 146) = 45.2, p < .001, R² = .48, adjusted R² = .47. Study hours\r\n(B = 1.80, SE = 0.31, β = .35, t = 5.78, p < .001, 95% CI [1.18, 2.42])\r\nand prior GPA (B = 8.52, SE = 1.95, β = .28, t = 4.37, p < .001,\r\n95% CI [4.66, 12.38]) were significant predictors, while attendance was\r\nnot (B = 0.15, SE = 0.12, β = .08, t = 1.25, p = .21, 95% CI [-0.09, 0.39]).\r\nMulticollinearity was not a concern (all VIF < 1.5).\r\n```\r\n\r\n#### Bayesian Analysis\r\n\r\n```\r\nA Bayesian independent samples t-test was conducted using weakly\r\ninformative priors (Normal(0, 1) for mean difference). The posterior\r\ndistribution indicated that Group A scored higher than Group B\r\n(M_diff = 6.8, 95% credible interval [3.2, 10.4]). The Bayes Factor\r\nBF₁₀ = 45.3 provided very strong evidence for a difference between\r\ngroups, with a 99.8% posterior probability that Group A's mean exceeded\r\nGroup B's mean. Convergence diagnostics were satisfactory (all R̂ < 1.01,\r\nESS > 1000).\r\n```\r\n\r\n---\r\n\r\n## Bayesian Statistics\r\n\r\n### When to Use Bayesian Methods\r\n\r\nConsider Bayesian approaches when:\r\n- You have prior information to incorporate\r\n- You want direct probability statements about hypotheses\r\n- Sample size is small or planning sequential data collection\r\n- You need to quantify evidence for the null hypothesis\r\n- The model is complex (hierarchical, missing data)\r\n\r\nSee `references/bayesian_statistics.md` for comprehensive guidance on:\r\n- Bayes' theorem and interpretation\r\n- Prior specification (informative, weakly informative, non-informative)\r\n- Bayesian hypothesis testing with Bayes Factors\r\n- Credible intervals vs. confidence intervals\r\n- Bayesian t-tests, ANOVA, regression, and hierarchical models\r\n- Model convergence checking and posterior predictive checks\r\n\r\n### Key Advantages\r\n\r\n1. **Intuitive interpretation**: \"Given the data, there is a 95% probability the parameter is in this interval\"\r\n2. **Evidence for null**: Can quantify support for no effect\r\n3. **Flexible**: No p-hacking concerns; can analyze data as it arrives\r\n4. **Uncertainty quantification**: Full posterior distribution\r\n\r\n---\r\n\r\n## Resources\r\n\r\nThis skill includes comprehensive reference materials:\r\n\r\n### References Directory\r\n\r\n- **test_selection_guide.md**: Decision tree for choosing appropriate statistical tests\r\n- **assumptions_and_diagnostics.md**: Detailed guidance on checking and handling assumption violations\r\n- **effect_sizes_and_power.md**: Calculating, interpreting, and reporting effect sizes; conducting power analyses\r\n- **bayesian_statistics.md**: Complete guide to Bayesian analysis methods\r\n- **reporting_standards.md**: APA-style reporting guidelines with examples\r\n\r\n### Scripts Directory\r\n\r\n- **assumption_checks.py**: Automated assumption checking with visualizations\r\n  - `comprehensive_assumption_check()`: Complete workflow\r\n  - `check_normality()`: Normality testing with Q-Q plots\r\n  - `check_homogeneity_of_variance()`: Levene's test with box plots\r\n  - `check_linearity()`: Regression linearity checks\r\n  - `detect_outliers()`: IQR and z-score outlier detection\r\n\r\n---\r\n\r\n## Best Practices\r\n\r\n1. **Pre-register analyses** when possible to distinguish confirmatory from exploratory\r\n2. **Always check assumptions** before interpreting results\r\n3. **Report effect sizes** with confidence intervals\r\n4. **Report all planned analyses** including non-significant results\r\n5. **Distinguish statistical from practical significance**\r\n6. **Visualize data** before and after analysis\r\n7. **Check diagnostics** for regression/ANOVA (residual plots, VIF, etc.)\r\n8. **Conduct sensitivity analyses** to assess robustness\r\n9. **Share data and code** for reproducibility\r\n10. **Be transparent** about violations, transformations, and decisions\r\n\r\n---\r\n\r\n## Common Pitfalls to Avoid\r\n\r\n1. **P-hacking**: Don't test multiple ways until something is significant\r\n2. **HARKing**: Don't present exploratory findings as confirmatory\r\n3. **Ignoring assumptions**: Check them and report violations\r\n4. **Confusing significance with importance**: p < .05 ≠ meaningful effect\r\n5. **Not reporting effect sizes**: Essential for interpretation\r\n6. **Cherry-picking results**: Report all planned analyses\r\n7. **Misinterpreting p-values**: They're NOT probability that hypothesis is true\r\n8. **Multiple comparisons**: Correct for family-wise error when appropriate\r\n9. **Ignoring missing data**: Understand mechanism (MCAR, MAR, MNAR)\r\n10. **Overinterpreting non-significant results**: Absence of evidence ≠ evidence of absence\r\n\r\n---\r\n\r\n## Getting Started Checklist\r\n\r\nWhen beginning a statistical analysis:\r\n\r\n- [ ] Define research question and hypotheses\r\n- [ ] Determine appropriate statistical test (use test_selection_guide.md)\r\n- [ ] Conduct power analysis to determine sample size\r\n- [ ] Load and inspect data\r\n- [ ] Check for missing data and outliers\r\n- [ ] Verify assumptions using assumption_checks.py\r\n- [ ] Run primary analysis\r\n- [ ] Calculate effect sizes with confidence intervals\r\n- [ ] Conduct post-hoc tests if needed (with corrections)\r\n- [ ] Create visualizations\r\n- [ ] Write results following reporting_standards.md\r\n- [ ] Conduct sensitivity analyses\r\n- [ ] Share data and code\r\n\r\n---\r\n\r\n## Support and Further Reading\r\n\r\nFor questions about:\r\n- **Test selection**: See references/test_selection_guide.md\r\n- **Assumptions**: See references/assumptions_and_diagnostics.md\r\n- **Effect sizes**: See references/effect_sizes_and_power.md\r\n- **Bayesian methods**: See references/bayesian_statistics.md\r\n- **Reporting**: See references/reporting_standards.md\r\n\r\n**Key textbooks**:\r\n- Cohen, J. (1988). *Statistical Power Analysis for the Behavioral Sciences*\r\n- Field, A. (2013). *Discovering Statistics Using IBM SPSS Statistics*\r\n- Gelman, A., & Hill, J. (2006). *Data Analysis Using Regression and Multilevel/Hierarchical Models*\r\n- Kruschke, J. K. (2014). *Doing Bayesian Data Analysis*\r\n\r\n**Online resources**:\r\n- APA Style Guide: https://apastyle.apa.org/\r\n- Statistical Consulting: Cross Validated (stats.stackexchange.com)"
  user_template: ""
variables: []
install:
  steps:
    - title: Download & unzip
      detail: >-
        Download the .zip package from SkillVault and unzip it. The statistical-analysis/ folder
        contains SKILL.md plus any companion scripts, references and assets.
    - title: Copy into your agent skills directory
      detail: >-
        Move the folder into your agent's skills directory — e.g. ~/.claude/skills/ (Claude Code),
        .codex/skills/, .cursor/skills/ or your project's .claude/skills/ folder — then restart or
        reload the session.
    - title: Use it
      detail: >-
        Describe the task in natural language; the agent loads the skill automatically when the
        request matches the skill description.
usage:
  steps: []
metadata:
  author: SkillVault
  source: https://skillvault.top/skills/statistical-analysis
  exported_at: "2026-11-03T07:42:11.579Z"