data-science-for-intelligence

The saved excerpt is a snapshot from review. The external source remains the complete and most current version.

---
name: data-science-for-intelligence
description: Statistical analysis, ML, NLP, time series forecasting, network analysis for political intelligence data
license: Apache-2.0
---

# Data Science for Intelligence Skill

## Purpose

This skill provides comprehensive data science methodologies tailored for political intelligence analysis in the CIA platform. It covers statistical analysis, machine learning, natural language processing, time series forecasting, and network analysis techniques applied to the 6 intelligence frameworks and 82 database views for democratic accountability assessment.

## When to Use This Skill

Apply this skill when:
- ✅ Building predictive models for election outcomes or coalition stability
- ✅ Analyzing temporal patterns in voting behavior or policy positions
- ✅ Detecting anomalies in political behavior (sudden voting shifts, absences)
- ✅ Performing text analysis on parliamentary documents and speeches
- ✅ Constructing influence networks from voting alignment patterns
- ✅ Forecasting trends in party support or politician effectiveness
- ✅ Clustering politicians or parties by voting similarity

Do NOT use for:
- ❌ Simple aggregation queries (use SQL views instead)
- ❌ Real-time operational dashboards (use materialized views)
- ❌ Causal inference without proper experimental design or quasi-experimental methods

## Data Science Framework for CIA Platform

### 6 Intelligence Analysis Frameworks

```mermaid
graph TB
    subgraph "Data Sources"
        A[Riksdagen API<br/>3.5M votes]
        B[Election Authority<br/>40 parties]
        C[World Bank<br/>598K indicators]
        D[Financial Authority<br/>Agency data]
    end
    
    subgraph "Data Science Techniques"
        A & B & C & D --> E[Feature Engineering]
        E --> F1[Time Series<br/>Analysis]
        E --> F2[Classification<br/>Models]
        E --> F3[Clustering<br/>Analysis]
        E --> F4[NLP<br/>Processing]
        E --> F5[Network<br/>Analysis]
    end
    
    subgraph "Intelligence Frameworks"
        F1 --> G1[1. Temporal<br/>Analysis]
        F2 --> G2[3. Pattern<br/>Recognition]
        F3 --> G3[2. Comparative<br/>Analysis]
        F4 --> G3
        F5 --> G5[5. Network<br/>Analysis]
        F1 & F2 --> G4[4. Predictive<br/>Intelligence]
        G1 & G2 & G3 & G4 & G5 --> G6[6. Decision<br/>Intelligence]
    end
    
    subgraph "Intelligence Products"
        G1 & G2 & G3 & G4 & G5 & G6 --> H[Risk Assessments]
        H --> I[Political Scorecards]
        H --> J[Coalition Forecasts]
        H --> K[Anomaly Alerts]
    end
    
    style E fill:#ffeb99
    style F1 fill:#e1f5ff
    style F2 fill:#e1f5ff
    style F3 fill:#e1f5ff
    style F4 fill:#e1f5ff
    style F5 fill:#e1f5ff
    style H fill:#ccffcc
```

## 1. Time Series Analysis (Temporal Framework)

**Purpose:** Analyze trends, seasonality, and forecast political metrics over time.

**CIA Platform Applications:**
- Politician voting participation trends
- Party support trajectories
- Government approval ratings
- Legislative productivity patterns

### Decomposition Analysis

**Example: Decompose Party Support into Trend, Seasonal, Residual Components**

```python
import pandas as pd
import numpy as np
from statsmodels.tsa.seasonal import seasonal_decompose
from statsmodels.tsa.stattools import adfuller
import matplotlib.pyplot as plt

class PoliticalTimeSeriesAnalyzer:
    """
    Time series analysis for political intelligence
    Supports: Temporal Analysis Framework
    """
    
    def __init__(self, db_connection):
        self.db = db_connection
    
    def decompose_party_support(self, party_code, election_years):
        """
        Decompose historical party support into trend, seasonal, residual
        
        Data Source: sweden_political_party table
        Intelligence Framework: Temporal Analysis
        """
        
        # Query: Historical election results
        query = """
        SELECT 
            election_year,
            percentage as