pytorch-lightning

# PyTorch Lightning

## Overview

PyTorch Lightning is a deep learning framework that organizes PyTorch code to eliminate boilerplate while maintaining full flexibility. Automate training workflows, multi-device orchestration, and implement best practices for neural network training and scaling across multiple GPUs/TPUs.

## When to Use This Skill

This skill should be used when:
- Building, training, or deploying neural networks using PyTorch Lightning
- Organizing PyTorch code into LightningModules
- Configuring Trainers for multi-GPU/TPU training
- Implementing data pipelines with LightningDataModules
- Working with callbacks, logging, and distributed training strategies (DDP, FSDP, DeepSpeed)
- Structuring deep learning projects professionally

## Core Capabilities

### 1. LightningModule - Model Definition

Organize PyTorch models into six logical sections:

1. **Initialization** - `__init__()` and `setup()`
2. **Training Loop** - `training_step(batch, batch_idx)`
3. **Validation Loop** - `validation_step(batch, batch_idx)`
4. **Test Loop** - `test_step(batch, batch_idx)`
5. **Prediction** - `predict_step(batch, batch_idx)`
6. **Optimizer Configuration** - `configure_optimizers()`

**Quick template reference:** See `scripts/template_lightning_module.py` for a complete boilerplate.

**Detailed documentation:** Read `references/lightning_module.md` for comprehensive method documentation, hooks, properties, and best practices.

### 2. Trainer - Training Automation

The Trainer automates the training loop, device management, gradient operations, and callbacks. Key features:

- Multi-GPU/TPU support with strategy selection (DDP, FSDP, DeepSpeed)
- Automatic mixed precision training
- Gradient accumulation and clipping
- Checkpointing and early stopping
- Progress bars and logging

**Quick setup reference:** See `scripts/quick_trainer_setup.py` for common Trainer configurations.

**Detailed documentation:** Read `references/trainer.md` for all parameters, methods, and configuration options.

### 3. LightningDataModule - Data Pipeline Organization

Encapsulate all data processing steps in a reusable class:

1. `prepare_data()` - Download and process data (single-process)
2. `setup()` - Create datasets and apply transforms (per-GPU)
3. `train_dataloader()` - Return training DataLoader
4. `val_dataloader()` - Return validation DataLoader
5. `test_dataloader()` - Return test DataLoader

**Quick template reference:** See `scripts/template_datamodule.py` for a complete boilerplate.

**Detailed documentation:** Read `references/data_module.md` for method details and usage patterns.

### 4. Callbacks - Extensible Training Logic

Add custom functionality at specific training hooks without modifying your LightningModule. Built-in callbacks include:

- **ModelCheckpoint** - Save best/latest models
- **EarlyStopping** - Stop when metrics plateau
- **LearningRateMonitor** - Track LR scheduler changes
- **BatchSizeFinder** - Auto-determine optimal batch size

**Detailed documentation:** Read `references/callbacks.md` for built-in callbacks and custom callback creation.

### 5. Logging - Experiment Tracking

Integrate with multiple logging platforms:

- TensorBoard (default)
- Weights & Biases (WandbLogger)
- MLflow (MLFlowLogger)
- Neptune (NeptuneLogger)
- Comet (CometLogger)
- CSV (CSVLogger)

Log metrics using `self.log("metric_name", value)` in any LightningModule method.

**Detailed documentation:** Read `references/logging.md` for logger setup and configuration.

### 6. Distributed Training - Scale to Multiple Devices

Choose the right strategy based on model size:

- **DDP** - For models <500M parameters (ResNet, smaller transformers)
- **FSDP** - For models 500M+ parameters (large transformers, recommended for Lightning users)
- **DeepSpeed** - For cutting-edge features and fine-grained control

Configure with: `Trainer(strategy="ddp", accelerator="gpu", devices=4)`

**Detailed documentation:** Read `references/distributed_training.md` for strategy comparison and configuration.

### 7. Best Practices

- Device agnostic code - Use `self.device` instead of `.cuda()`
- Hyperparameter saving - Use `self.save_hyperparameters()` in `__init__()`
- Metric logging - Use `self.log()` for automatic aggregation across devices
- Reproducibility - Use `seed_everything()` and `Trainer(deterministic=True)`
- Debugging - Use `Trainer(fast_dev_run=True)` to test with 1 batch

**Detailed documentation:** Read `references/best_practices.md` for common patterns and pitfalls.

## Quick Workflow

1. **Define model:**
   ```python
   class MyModel(L.LightningModule):
       def __init__(self):
           super().__init__()
           self.save_hyperparameters()
           self.model = YourNetwork()

       def training_step(self, batch, batch_idx):
           x, y = batch
           loss = F.cross_entropy(self.model(x), y)
           self.log("train_loss", loss)
           return loss

       def configure_optimizers(self):
           return torch.optim.Adam(self.parameters())
   ```

2. **Prepare data:**
   ```python
   # Option 1: Direct DataLoaders
   train_loader = DataLoader(train_dataset, batch_size=32)

   # Option 2: LightningDataModule (recommended for reusability)
   dm = MyDataModule(batch_size=32)
   ```

3. **Train:**
   ```python
   trainer = L.Trainer(max_epochs=10, accelerator="gpu", devices=2)
   trainer.fit(model, train_loader)  # or trainer.fit(model, datamodule=dm)
   ```

## Resources

### scripts/
Executable Python templates for common PyTorch Lightning patterns:

- `template_lightning_module.py` - Complete LightningModule boilerplate
- `template_datamodule.py` - Complete LightningDataModule boilerplate
- `quick_trainer_setup.py` - Common Trainer configuration examples

### references/
Detailed documentation for each PyTorch Lightning component:

- `lightning_module.md` - Comprehensive LightningModule guide (methods, hooks, properties)
- `trainer.md` - Trainer configuration and parameters
- `data_module.md` - LightningDataModule patterns and methods
- `callbacks.md` - Built-in and custom callbacks
- `logging.md` - Logger integrations and usage
- `distributed_training.md` - DDP, FSDP, DeepSpeed comparison and setup
- `best_practices.md` - Common patterns, tips, and pitfalls

{
  "format": "aiskill/v1",
  "skill": {
    "name": "pytorch-lightning",
    "slug": "pytorch-lightning",
    "version": "1.0.0",
    "description": "Deep learning framework (PyTorch Lightning). Organize PyTorch code into LightningModules, configure Trainers for multi-GPU/TPU, implement data pipelines, callbacks, logging (W&B, TensorBoard), distributed training (DDP, FSDP, DeepSpeed), for scalable neural network training.",
    "category": "Research",
    "tags": [
      "research-science"
    ],
    "platforms": [
      "chatgpt",
      "claude-code",
      "cursor"
    ],
    "tested_environment": "Agent Skills convention (SKILL.md) — works with Claude Code, Cursor, ChatGPT/Codex and compatible agents",
    "license": "APACHE-2.0 LICENSE"
  },
  "prompts": {
    "system": "# PyTorch Lightning\r\n\r\n## Overview\r\n\r\nPyTorch Lightning is a deep learning framework that organizes PyTorch code to eliminate boilerplate while maintaining full flexibility. Automate training workflows, multi-device orchestration, and implement best practices for neural network training and scaling across multiple GPUs/TPUs.\r\n\r\n## When to Use This Skill\r\n\r\nThis skill should be used when:\r\n- Building, training, or deploying neural networks using PyTorch Lightning\r\n- Organizing PyTorch code into LightningModules\r\n- Configuring Trainers for multi-GPU/TPU training\r\n- Implementing data pipelines with LightningDataModules\r\n- Working with callbacks, logging, and distributed training strategies (DDP, FSDP, DeepSpeed)\r\n- Structuring deep learning projects professionally\r\n\r\n## Core Capabilities\r\n\r\n### 1. LightningModule - Model Definition\r\n\r\nOrganize PyTorch models into six logical sections:\r\n\r\n1. **Initialization** - `__init__()` and `setup()`\r\n2. **Training Loop** - `training_step(batch, batch_idx)`\r\n3. **Validation Loop** - `validation_step(batch, batch_idx)`\r\n4. **Test Loop** - `test_step(batch, batch_idx)`\r\n5. **Prediction** - `predict_step(batch, batch_idx)`\r\n6. **Optimizer Configuration** - `configure_optimizers()`\r\n\r\n**Quick template reference:** See `scripts/template_lightning_module.py` for a complete boilerplate.\r\n\r\n**Detailed documentation:** Read `references/lightning_module.md` for comprehensive method documentation, hooks, properties, and best practices.\r\n\r\n### 2. Trainer - Training Automation\r\n\r\nThe Trainer automates the training loop, device management, gradient operations, and callbacks. Key features:\r\n\r\n- Multi-GPU/TPU support with strategy selection (DDP, FSDP, DeepSpeed)\r\n- Automatic mixed precision training\r\n- Gradient accumulation and clipping\r\n- Checkpointing and early stopping\r\n- Progress bars and logging\r\n\r\n**Quick setup reference:** See `scripts/quick_trainer_setup.py` for common Trainer configurations.\r\n\r\n**Detailed documentation:** Read `references/trainer.md` for all parameters, methods, and configuration options.\r\n\r\n### 3. LightningDataModule - Data Pipeline Organization\r\n\r\nEncapsulate all data processing steps in a reusable class:\r\n\r\n1. `prepare_data()` - Download and process data (single-process)\r\n2. `setup()` - Create datasets and apply transforms (per-GPU)\r\n3. `train_dataloader()` - Return training DataLoader\r\n4. `val_dataloader()` - Return validation DataLoader\r\n5. `test_dataloader()` - Return test DataLoader\r\n\r\n**Quick template reference:** See `scripts/template_datamodule.py` for a complete boilerplate.\r\n\r\n**Detailed documentation:** Read `references/data_module.md` for method details and usage patterns.\r\n\r\n### 4. Callbacks - Extensible Training Logic\r\n\r\nAdd custom functionality at specific training hooks without modifying your LightningModule. Built-in callbacks include:\r\n\r\n- **ModelCheckpoint** - Save best/latest models\r\n- **EarlyStopping** - Stop when metrics plateau\r\n- **LearningRateMonitor** - Track LR scheduler changes\r\n- **BatchSizeFinder** - Auto-determine optimal batch size\r\n\r\n**Detailed documentation:** Read `references/callbacks.md` for built-in callbacks and custom callback creation.\r\n\r\n### 5. Logging - Experiment Tracking\r\n\r\nIntegrate with multiple logging platforms:\r\n\r\n- TensorBoard (default)\r\n- Weights & Biases (WandbLogger)\r\n- MLflow (MLFlowLogger)\r\n- Neptune (NeptuneLogger)\r\n- Comet (CometLogger)\r\n- CSV (CSVLogger)\r\n\r\nLog metrics using `self.log(\"metric_name\", value)` in any LightningModule method.\r\n\r\n**Detailed documentation:** Read `references/logging.md` for logger setup and configuration.\r\n\r\n### 6. Distributed Training - Scale to Multiple Devices\r\n\r\nChoose the right strategy based on model size:\r\n\r\n- **DDP** - For models <500M parameters (ResNet, smaller transformers)\r\n- **FSDP** - For models 500M+ parameters (large transformers, recommended for Lightning users)\r\n- **DeepSpeed** - For cutting-edge features and fine-grained control\r\n\r\nConfigure with: `Trainer(strategy=\"ddp\", accelerator=\"gpu\", devices=4)`\r\n\r\n**Detailed documentation:** Read `references/distributed_training.md` for strategy comparison and configuration.\r\n\r\n### 7. Best Practices\r\n\r\n- Device agnostic code - Use `self.device` instead of `.cuda()`\r\n- Hyperparameter saving - Use `self.save_hyperparameters()` in `__init__()`\r\n- Metric logging - Use `self.log()` for automatic aggregation across devices\r\n- Reproducibility - Use `seed_everything()` and `Trainer(deterministic=True)`\r\n- Debugging - Use `Trainer(fast_dev_run=True)` to test with 1 batch\r\n\r\n**Detailed documentation:** Read `references/best_practices.md` for common patterns and pitfalls.\r\n\r\n## Quick Workflow\r\n\r\n1. **Define model:**\r\n   ```python\r\n   class MyModel(L.LightningModule):\r\n       def __init__(self):\r\n           super().__init__()\r\n           self.save_hyperparameters()\r\n           self.model = YourNetwork()\r\n\r\n       def training_step(self, batch, batch_idx):\r\n           x, y = batch\r\n           loss = F.cross_entropy(self.model(x), y)\r\n           self.log(\"train_loss\", loss)\r\n           return loss\r\n\r\n       def configure_optimizers(self):\r\n           return torch.optim.Adam(self.parameters())\r\n   ```\r\n\r\n2. **Prepare data:**\r\n   ```python\r\n   # Option 1: Direct DataLoaders\r\n   train_loader = DataLoader(train_dataset, batch_size=32)\r\n\r\n   # Option 2: LightningDataModule (recommended for reusability)\r\n   dm = MyDataModule(batch_size=32)\r\n   ```\r\n\r\n3. **Train:**\r\n   ```python\r\n   trainer = L.Trainer(max_epochs=10, accelerator=\"gpu\", devices=2)\r\n   trainer.fit(model, train_loader)  # or trainer.fit(model, datamodule=dm)\r\n   ```\r\n\r\n## Resources\r\n\r\n### scripts/\r\nExecutable Python templates for common PyTorch Lightning patterns:\r\n\r\n- `template_lightning_module.py` - Complete LightningModule boilerplate\r\n- `template_datamodule.py` - Complete LightningDataModule boilerplate\r\n- `quick_trainer_setup.py` - Common Trainer configuration examples\r\n\r\n### references/\r\nDetailed documentation for each PyTorch Lightning component:\r\n\r\n- `lightning_module.md` - Comprehensive LightningModule guide (methods, hooks, properties)\r\n- `trainer.md` - Trainer configuration and parameters\r\n- `data_module.md` - LightningDataModule patterns and methods\r\n- `callbacks.md` - Built-in and custom callbacks\r\n- `logging.md` - Logger integrations and usage\r\n- `distributed_training.md` - DDP, FSDP, DeepSpeed comparison and setup\r\n- `best_practices.md` - Common patterns, tips, and pitfalls",
    "user_template": ""
  },
  "variables": [],
  "install": {
    "steps": [
      {
        "title": "Download & unzip",
        "detail": "Download the .zip package from SkillVault and unzip it. The pytorch-lightning/ folder contains SKILL.md plus any companion scripts, references and assets."
      },
      {
        "title": "Copy into your agent skills directory",
        "detail": "Move the folder into your agent's skills directory — e.g. ~/.claude/skills/ (Claude Code), .codex/skills/, .cursor/skills/ or your project's .claude/skills/ folder — then restart or reload the session."
      },
      {
        "title": "Use it",
        "detail": "Describe the task in natural language; the agent loads the skill automatically when the request matches the skill description."
      }
    ]
  },
  "usage": {
    "steps": []
  },
  "metadata": {
    "author": "SkillVault",
    "source": "https://skillvault.top/skills/pytorch-lightning",
    "exported_at": "2026-11-03T07:42:10.674Z"
  }
}

format: aiskill/v1
skill:
  name: pytorch-lightning
  slug: pytorch-lightning
  version: 1.0.0
  description: >-
    Deep learning framework (PyTorch Lightning). Organize PyTorch code into LightningModules,
    configure Trainers for multi-GPU/TPU, implement data pipelines, callbacks, logging (W&B,
    TensorBoard), distributed training (DDP, FSDP, DeepSpeed), for scalable neural network training.
  category: Research
  tags:
    - research-science
  platforms:
    - chatgpt
    - claude-code
    - cursor
  tested_environment: >-
    Agent Skills convention (SKILL.md) — works with Claude Code, Cursor, ChatGPT/Codex and
    compatible agents
  license: APACHE-2.0 LICENSE
prompts:
  system: "# PyTorch Lightning\r\n\r\n## Overview\r\n\r\nPyTorch Lightning is a deep learning framework that organizes PyTorch code to eliminate boilerplate while maintaining full flexibility. Automate training workflows, multi-device orchestration, and implement best practices for neural network training and scaling across multiple GPUs/TPUs.\r\n\r\n## When to Use This Skill\r\n\r\nThis skill should be used when:\r\n- Building, training, or deploying neural networks using PyTorch Lightning\r\n- Organizing PyTorch code into LightningModules\r\n- Configuring Trainers for multi-GPU/TPU training\r\n- Implementing data pipelines with LightningDataModules\r\n- Working with callbacks, logging, and distributed training strategies (DDP, FSDP, DeepSpeed)\r\n- Structuring deep learning projects professionally\r\n\r\n## Core Capabilities\r\n\r\n### 1. LightningModule - Model Definition\r\n\r\nOrganize PyTorch models into six logical sections:\r\n\r\n1. **Initialization** - `__init__()` and `setup()`\r\n2. **Training Loop** - `training_step(batch, batch_idx)`\r\n3. **Validation Loop** - `validation_step(batch, batch_idx)`\r\n4. **Test Loop** - `test_step(batch, batch_idx)`\r\n5. **Prediction** - `predict_step(batch, batch_idx)`\r\n6. **Optimizer Configuration** - `configure_optimizers()`\r\n\r\n**Quick template reference:** See `scripts/template_lightning_module.py` for a complete boilerplate.\r\n\r\n**Detailed documentation:** Read `references/lightning_module.md` for comprehensive method documentation, hooks, properties, and best practices.\r\n\r\n### 2. Trainer - Training Automation\r\n\r\nThe Trainer automates the training loop, device management, gradient operations, and callbacks. Key features:\r\n\r\n- Multi-GPU/TPU support with strategy selection (DDP, FSDP, DeepSpeed)\r\n- Automatic mixed precision training\r\n- Gradient accumulation and clipping\r\n- Checkpointing and early stopping\r\n- Progress bars and logging\r\n\r\n**Quick setup reference:** See `scripts/quick_trainer_setup.py` for common Trainer configurations.\r\n\r\n**Detailed documentation:** Read `references/trainer.md` for all parameters, methods, and configuration options.\r\n\r\n### 3. LightningDataModule - Data Pipeline Organization\r\n\r\nEncapsulate all data processing steps in a reusable class:\r\n\r\n1. `prepare_data()` - Download and process data (single-process)\r\n2. `setup()` - Create datasets and apply transforms (per-GPU)\r\n3. `train_dataloader()` - Return training DataLoader\r\n4. `val_dataloader()` - Return validation DataLoader\r\n5. `test_dataloader()` - Return test DataLoader\r\n\r\n**Quick template reference:** See `scripts/template_datamodule.py` for a complete boilerplate.\r\n\r\n**Detailed documentation:** Read `references/data_module.md` for method details and usage patterns.\r\n\r\n### 4. Callbacks - Extensible Training Logic\r\n\r\nAdd custom functionality at specific training hooks without modifying your LightningModule. Built-in callbacks include:\r\n\r\n- **ModelCheckpoint** - Save best/latest models\r\n- **EarlyStopping** - Stop when metrics plateau\r\n- **LearningRateMonitor** - Track LR scheduler changes\r\n- **BatchSizeFinder** - Auto-determine optimal batch size\r\n\r\n**Detailed documentation:** Read `references/callbacks.md` for built-in callbacks and custom callback creation.\r\n\r\n### 5. Logging - Experiment Tracking\r\n\r\nIntegrate with multiple logging platforms:\r\n\r\n- TensorBoard (default)\r\n- Weights & Biases (WandbLogger)\r\n- MLflow (MLFlowLogger)\r\n- Neptune (NeptuneLogger)\r\n- Comet (CometLogger)\r\n- CSV (CSVLogger)\r\n\r\nLog metrics using `self.log(\"metric_name\", value)` in any LightningModule method.\r\n\r\n**Detailed documentation:** Read `references/logging.md` for logger setup and configuration.\r\n\r\n### 6. Distributed Training - Scale to Multiple Devices\r\n\r\nChoose the right strategy based on model size:\r\n\r\n- **DDP** - For models <500M parameters (ResNet, smaller transformers)\r\n- **FSDP** - For models 500M+ parameters (large transformers, recommended for Lightning users)\r\n- **DeepSpeed** - For cutting-edge features and fine-grained control\r\n\r\nConfigure with: `Trainer(strategy=\"ddp\", accelerator=\"gpu\", devices=4)`\r\n\r\n**Detailed documentation:** Read `references/distributed_training.md` for strategy comparison and configuration.\r\n\r\n### 7. Best Practices\r\n\r\n- Device agnostic code - Use `self.device` instead of `.cuda()`\r\n- Hyperparameter saving - Use `self.save_hyperparameters()` in `__init__()`\r\n- Metric logging - Use `self.log()` for automatic aggregation across devices\r\n- Reproducibility - Use `seed_everything()` and `Trainer(deterministic=True)`\r\n- Debugging - Use `Trainer(fast_dev_run=True)` to test with 1 batch\r\n\r\n**Detailed documentation:** Read `references/best_practices.md` for common patterns and pitfalls.\r\n\r\n## Quick Workflow\r\n\r\n1. **Define model:**\r\n   ```python\r\n   class MyModel(L.LightningModule):\r\n       def __init__(self):\r\n           super().__init__()\r\n           self.save_hyperparameters()\r\n           self.model = YourNetwork()\r\n\r\n       def training_step(self, batch, batch_idx):\r\n           x, y = batch\r\n           loss = F.cross_entropy(self.model(x), y)\r\n           self.log(\"train_loss\", loss)\r\n           return loss\r\n\r\n       def configure_optimizers(self):\r\n           return torch.optim.Adam(self.parameters())\r\n   ```\r\n\r\n2. **Prepare data:**\r\n   ```python\r\n   # Option 1: Direct DataLoaders\r\n   train_loader = DataLoader(train_dataset, batch_size=32)\r\n\r\n   # Option 2: LightningDataModule (recommended for reusability)\r\n   dm = MyDataModule(batch_size=32)\r\n   ```\r\n\r\n3. **Train:**\r\n   ```python\r\n   trainer = L.Trainer(max_epochs=10, accelerator=\"gpu\", devices=2)\r\n   trainer.fit(model, train_loader)  # or trainer.fit(model, datamodule=dm)\r\n   ```\r\n\r\n## Resources\r\n\r\n### scripts/\r\nExecutable Python templates for common PyTorch Lightning patterns:\r\n\r\n- `template_lightning_module.py` - Complete LightningModule boilerplate\r\n- `template_datamodule.py` - Complete LightningDataModule boilerplate\r\n- `quick_trainer_setup.py` - Common Trainer configuration examples\r\n\r\n### references/\r\nDetailed documentation for each PyTorch Lightning component:\r\n\r\n- `lightning_module.md` - Comprehensive LightningModule guide (methods, hooks, properties)\r\n- `trainer.md` - Trainer configuration and parameters\r\n- `data_module.md` - LightningDataModule patterns and methods\r\n- `callbacks.md` - Built-in and custom callbacks\r\n- `logging.md` - Logger integrations and usage\r\n- `distributed_training.md` - DDP, FSDP, DeepSpeed comparison and setup\r\n- `best_practices.md` - Common patterns, tips, and pitfalls"
  user_template: ""
variables: []
install:
  steps:
    - title: Download & unzip
      detail: >-
        Download the .zip package from SkillVault and unzip it. The pytorch-lightning/ folder
        contains SKILL.md plus any companion scripts, references and assets.
    - title: Copy into your agent skills directory
      detail: >-
        Move the folder into your agent's skills directory — e.g. ~/.claude/skills/ (Claude Code),
        .codex/skills/, .cursor/skills/ or your project's .claude/skills/ folder — then restart or
        reload the session.
    - title: Use it
      detail: >-
        Describe the task in natural language; the agent loads the skill automatically when the
        request matches the skill description.
usage:
  steps: []
metadata:
  author: SkillVault
  source: https://skillvault.top/skills/pytorch-lightning
  exported_at: "2026-11-03T07:42:10.674Z"