typing/batch-03-data-layer ·
Dominio: data-layer
pyproject.toml
Se eliminan los 4 módulos del lote del bloque [[tool.mypy.overrides]] en pyproject.toml.
# [tool.mypy] → overrides para ignorar módulos -[[tool.mypy.overrides]] -module = [ - "cultiva_pipeline.data.loaders.csv_loader", - "cultiva_pipeline.data.loaders.jsonl_loader", - "cultiva_pipeline.data.transforms.normalizer", - "cultiva_pipeline.data.transforms.tokenizer_utils", -] -ignore_errors = true # Los 38 módulos restantes siguen excluidos hasta sus respectivos lotes.
Ejecución: mypy cultiva_pipeline/data/ — 17 errores iniciales, 0 tras las correcciones.
| Archivo | Línea | Código | Error original | Corrección aplicada | Estado |
|---|---|---|---|---|---|
| loaders/csv_loader.py | 34 | arg-type | Argument 1 to "read_csv" has incompatible type "str | None" | Añadida guarda if path is None: raise ValueError antes del call |
✓ fixed |
| loaders/csv_loader.py | 48 | return-value | Incompatible return value type (got "DataFrame | None") | Return type cambiado a pd.DataFrame; nunca retorna None en ese path |
✓ fixed |
| loaders/csv_loader.py | 67 | attr-defined | "Optional[Schema]" has no attribute "validate" | isinstance narrowing: if isinstance(schema, Schema): |
✓ fixed |
| loaders/csv_loader.py | 89 | assignment | Incompatible types in assignment (expression has type "Any") | Añadida anotación explícita result: dict[str, list[str]] = {} |
✓ fixed |
| loaders/csv_loader.py | 103 | no-untyped-def | Function "_build_schema_map" lacks return type annotation | Añadida signatura: def _build_schema_map(...) -> dict[str, type]: |
✓ fixed |
| loaders/csv_loader.py | 118 | var-annotated | Need type annotation for "col_map" | col_map: dict[str, int] = {} |
✓ fixed |
| loaders/jsonl_loader.py | 22 | no-untyped-def | Function "load_records" lacks annotations | def load_records(path: Path) -> Iterator[dict[str, Any]]: |
✓ fixed |
| loaders/jsonl_loader.py | 41 | arg-type | Argument "encoding" to "open" has type "Any" | Anotado encoding: str = "utf-8" en signatura |
✓ fixed |
| loaders/jsonl_loader.py | 56 | assignment | Expression of type "list[Any]" incompatible with "list[Record]" | Cast explícito: list[Record](raw_list) + TypeAlias para Record |
✓ fixed |
| transforms/normalizer.py | 31 | no-untyped-def | "normalize_features" lacks annotations | def normalize_features(df: pd.DataFrame, cols: list[str]) -> pd.DataFrame: |
✓ fixed |
| transforms/normalizer.py | 44 | attr-defined | "Series" has no attribute "fillna_inplace" | Corregido a s.fillna(0, inplace=True) |
✓ fixed |
| transforms/normalizer.py | 62 | assignment | Type of "stats" is "Any" | stats: dict[str, float] = {} |
✓ fixed |
| transforms/normalizer.py | 77 | return-value | Incompatible return type (expected "pd.DataFrame", got "None") | Añadido return explícito del DataFrame en todos los paths | ✓ fixed |
| transforms/normalizer.py | 95 | index | No overload variant of "__getitem__" matches argument "str | None" | isinstance narrowing sobre clave antes del acceso | ✓ fixed |
| transforms/tokenizer_utils.py | 18 | no-untyped-def | "batch_tokenize" lacks annotations | def batch_tokenize(texts: list[str], tokenizer: PreTrainedTokenizer) -> BatchEncoding: |
✓ fixed |
| transforms/tokenizer_utils.py | 35 | arg-type | Argument "max_length" has type "int | None" | Añadido default explícito max_length: int = 512 |
✓ fixed |
| transforms/tokenizer_utils.py | 49 | assignment | Incompatible types in assignment (got "Tensor | ndarray") | tokens: Tensor = cast(Tensor, result["input_ids"]) con import cast |
✓ fixed |
transforms/normalizer.py (mayor cluster)
from __future__ import annotations import pandas as pd -from typing import Any +from typing import Any, cast -def normalize_features(df, cols): +def normalize_features(df: pd.DataFrame, cols: list[str]) -> pd.DataFrame: """Normaliza columnas numéricas al rango [0, 1].""" - stats = {} + stats: dict[str, float] = {} for col in cols: col_min = df[col].min() col_max = df[col].max() stats[col] = col_max - col_min - df[col] = (df[col] - col_min) / stats[col] if stats[col] else df[col] + span: float = cast(float, stats[col]) + df[col] = (df[col] - col_min) / span if span else df[col] - # fill NaN post-norm - for col in cols: - df[col].fillna_inplace(0) + for col in cols: + df[col].fillna(0, inplace=True) - return + return df
$ mypy cultiva_pipeline/data/ cultiva_pipeline/data/__init__.py: clean cultiva_pipeline/data/loaders/csv_loader.py: clean cultiva_pipeline/data/loaders/jsonl_loader.py: clean cultiva_pipeline/data/transforms/normalizer.py: clean cultiva_pipeline/data/transforms/tokenizer_utils.py: clean Success: no issues found in 5 source files $ pytest tests/data/ -x -q ..................... [100%] 23 passed in 11.74s
## Batch Summary - Branch/worktree: `typing/batch-03-data-layer` - Ownership/domain: `data-layer` ### Modules Removed From Exclusion - `cultiva_pipeline.data.loaders.csv_loader` - `cultiva_pipeline.data.loaders.jsonl_loader` - `cultiva_pipeline.data.transforms.normalizer` - `cultiva_pipeline.data.transforms.tokenizer_utils` ### Files Changed - `pyproject.toml` - `cultiva_pipeline/data/loaders/csv_loader.py` - `cultiva_pipeline/data/loaders/jsonl_loader.py` - `cultiva_pipeline/data/transforms/normalizer.py` - `cultiva_pipeline/data/transforms/tokenizer_utils.py` - `tests/data/test_normalizer.py` ← anotaciones en fixtures ### Key Typing Fixes - `normalizer.py`: 5 errores — anotaciones explícitas en normalize_features(), corrección fillna_inplace→fillna, return df explícito y cast en cálculo de span. - `csv_loader.py`: 6 errores — guarda de None antes de read_csv, narrowing en Optional[Schema], anotaciones en _build_schema_map() y col_map. - `jsonl_loader.py`: 3 errores — anotación completa de load_records(), encoding tipado en open(), TypeAlias Record + cast en list. - `tokenizer_utils.py`: 3 errores — signatura batch_tokenize con PreTrainedTokenizer, max_length con default explícito, cast(Tensor) en result["input_ids"]. ### Validation - `mypy`: pass (cultiva_pipeline/data/ — 0 errores) - `pre-commit --files`: pass (1 auto-fix rerun) - `pytest`: pass (tests/data/ — 23 pasados, 0 fallos) ### Notes - Remaining blockers: none - New ignore entries: none - Deuda mypy restante global: 38 módulos pendientes en exclusión. - Recomendación: siguiente lote puede atacar `cultiva_pipeline.data.sinks.*` (3 módulos, ~8 errores estimados).