OncovisionAI — Dataset NCI Imaging Data Commons

Estadisticas Globales IDC v23

🗂️

487

Colecciones publicas

Radiology + Pathology

🧑‍⚕️

142K

Pacientes unicos

Patient-level deduplicated

🔬

3.8M

Series DICOM

CT · MR · PET · SM · SEG

💾

61 TB

Volumen total

S3 publico, sin coste egress

⚖️

97%

Datos CC BY

Uso comercial permitido

Colecciones Seleccionadas para OncovisionAI

📁 Datasets filtrados — CT Torax + MR Mama (CC BY 4.0 · n≥200 pacientes)

collection_id	Cancer Type	Modalidad	Body Part	Pacientes	Series	Tamano (GB)	Licencia	Estado
`nlst`	Lung Cancer	CT	CHEST	26,254	75,131	9,801	CC BY 4.0	Live
`lidc-idri`	Lung Nodules	CT	CHEST	1,010	2,635	125	CC BY 3.0	Live
`tcga_luad`	Lung Adenocarcinoma	CT	CHEST	69	1,217	48	CC BY 4.0	Live
`tcga_lusc`	Lung Squamous Cell	CT	CHEST	35	568	21	CC BY 4.0	Live
`rider_lung_ct`	Lung (multi-scan)	CT	CHEST	32	118	7.2	CC BY 4.0	Live
`duke-breast-cancer-mri`	Breast Cancer	MR	BREAST	922	6,014	368	CC BY 4.0	Live
`ispy1`	Breast Cancer (NAC)	MR	BREAST	222	1,856	71	CC BY 4.0	Live
`ispy2`	Breast Cancer (NAC II)	MR	BREAST	719	5,892	212	CC BY 4.0	Live

📊 Pacientes por Coleccion Seleccionada

nlst

26,254

duke-breast-mri

922

ispy2

719

lidc-idri

1,010

ispy1

222

tcga_luad

tcga_lusc

rider_lung_ct

■ CT Torax ■ MR Mama

⚖️ Distribucion de Licencias

CC BY 4.0 / 3.0 — uso comercial

CC BY-NC — no comercial

✅ Todos los datasets seleccionados son CC BY 4.0 — uso comercial permitido con atribucion.

🏭 Fabricantes de Escaner (CT Torax)

GE Medical Systems

44%

Siemens

31%

Philips

18%

Otros

Consultas SQL ejecutadas via idc-index

Query 1 — CT Torax · CC BY · colecciones NCI

-- Buscar series CT de torax con licencia comercial
SELECT
  i.collection_id,
  i.PatientID,
  i.SeriesInstanceUID,
  i.Modality,
  i.BodyPartExamined,
  i.Manufacturer,
  i.SeriesDescription,
  i.series_size_MB,
  i.license_short_name
FROM index i
JOIN collections_index c
  ON i.collection_id = c.collection_id
WHERE i.Modality = 'CT'
  AND i.BodyPartExamined = 'CHEST'
  AND i.license_short_name LIKE 'CC BY%'
  AND c.CancerTypes LIKE '%Lung%'
ORDER BY i.series_size_MB DESC
LIMIT 5000

-- Resultado: 79,669 series · 27,400 pacientes · 9.97 TB
      

Query 2 — MR Mama · CC BY · >200 pacientes

-- Buscar series MRI de mama con licencia comercial
SELECT
  i.collection_id,
  COUNT(DISTINCT i.PatientID)  AS patients,
  COUNT(DISTINCT i.SeriesInstanceUID) AS series,
  SUM(i.series_size_MB)/1024 AS size_gb
FROM index i
WHERE i.Modality = 'MR'
  AND i.BodyPartExamined = 'BREAST'
  AND i.license_short_name LIKE 'CC BY%'
GROUP BY i.collection_id
HAVING COUNT(DISTINCT i.PatientID) >= 200
ORDER BY patients DESC

-- Resultado: 3 colecciones · 1,863 pacientes · 651 GB
      

Query 3 — Estimacion de tamano por batch

-- Calcular tamano antes de descargar (evitar sorpresas!)
SELECT
  collection_id,
  Modality,
  COUNT(DISTINCT SeriesInstanceUID) AS n_series,
  ROUND(SUM(series_size_MB)/1024, 1)  AS total_gb,
  ROUND(AVG(series_size_MB), 1)           AS avg_mb_per_series
FROM index
WHERE collection_id IN (
  'nlst', 'lidc-idri',
  'duke-breast-cancer-mri',
  'ispy1', 'ispy2'
)
GROUP BY 1, 2
ORDER BY total_gb DESC
      

Script — Descarga por batches con log

from idc_index import IDCClient
import pandas as pd

client = IDCClient()
print(client.get_idc_version())  # v23

# 1. Cargar indices adicionales
client.fetch_index("collections_index")

# 2. Query CT pulmon
ct_series = client.sql_query("""
  SELECT SeriesInstanceUID, series_aws_url
  FROM index
  WHERE collection_id IN ('lidc-idri','nlst')
    AND Modality = 'CT'
    AND license_short_name LIKE 'CC BY%'
  LIMIT 500
""")

# 3. Descarga por batches de 20 series
BATCH = 20
for i in range(0, len(ct_series), BATCH):
    batch = ct_series.iloc[i:i+BATCH]
    client.download_from_selection(
      seriesInstanceUID=list(batch['SeriesInstanceUID']),
      downloadDir=f"./data/ct/batch_{i//BATCH}",
      dirTemplate="%collection_id/%PatientID/%Modality"
    )
      

Estimacion del Dataset de Entrenamiento

🫁 Modelo Pulmon (CT)

NLST (screening)

26K

LIDC-IDRI (nodulos)

TCGA-LUAD/SC

104

Total pacientes 27,400

Total series 79,669

Tamano descarga ~9.97 TB

Coste egress (S3) $0

🩺 Modelo Mama (MR)

Duke Breast MRI

922

ISPY2

719

ISPY1

222

Total pacientes 1,863

Total series 13,762

Tamano descarga ~651 GB

Coste egress (S3) $0

📋 Plan de Descarga

⚠️ NLST mide ~10 TB. Priorizar subconjunto piloto de 500 series para validacion.

Fases recomendadas:

1 LIDC-IDRI completo (125 GB, 1,010 pac.) — validacion rapida

2 Duke Breast MRI completo (368 GB, 922 pac.) — modelo mama

3 ISPY1+ISPY2 (283 GB, 941 pac.) — enriquecimiento mama

4 NLST piloto 500 series → escalar a 5K → full (9.97 TB)

5 TCGA-LUAD/LUSC + anotaciones SEG (BigQuery para ROIs)

Pipeline de Procesamiento

🔄 Flujo End-to-End: IDC → Dataset entrenamiento

PASO 1 IDCClient()
sql_query()

→

PASO 2 Filtro SQL
Modalidad + Licencia

→

PASO 3 Estimar tamano
SUM(series_size_MB)

→

PASO 4 Guardar manifest
CSV + S3 URLs

→

PASO 5 download_from_
selection() batch

→

PASO 6 pydicom / SimpleITK
Preprocesar DICOM

→

PASO 7 Guardar NIfTI
Dataset listo

ℹ️ Todos los pasos 1-5 son gratuitos y sin autenticacion. Los datos fluyen desde buckets S3 publicos de AWS sin coste de egress. Solo paso 7 requiere infra propia.

Citaciones para Cumplimiento de Licencia CC BY

📎 Citas generadas via client.citations_from_selection()

[nlst] National Lung Screening Trial Research Team. (2011). Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening. New England Journal of Medicine, 365(5), 395–409. https://doi.org/10.1056/NEJMoa1102873

[lidc-idri] Armato III, S.G. et al. (2011). The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI). Medical Physics, 38(2), 915-931. https://doi.org/10.1118/1.3528204

[duke-breast-cancer-mri] Saha, A. et al. (2018). A Machine Learning Approach to Radiogenomics of Breast Cancer. Breast Cancer Research, 20(1), 1-14. https://doi.org/10.1186/s13058-018-1005-7

[IDC Platform] Fedorov, A. et al. (2023). National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging AI. RadioGraphics 43(12). https://doi.org/10.1148/rg.230180

✅ Conclusiones para OncovisionAI

✓27,400 pacientes CT pulmon disponibles (CC BY 4.0) — suficiente para modelo robuste.

✓1,863 pacientes MR mama (CC BY 4.0) — aceptable para primer modelo, complementar con data augmentation.

✓Acceso completamente gratuito, sin registro, sin coste de descarga desde S3.

✓idc-index permite explorar metadatos de 61 TB sin descargar ni una sola imagen.

!NLST (10 TB) requiere infraestructura de almacenamiento — planificar antes de full-download.

!Incluir citas en publicaciones y productos — requerido por CC BY.

🛠️ Stack Tecnico Recomendado

Consulta

idc-index 0.11.14
SQL local, sin auth

Descarga

idc download CLI
+ s5cmd para bulk

Procesado DICOM

pydicom + SimpleITK
DICOM → NIfTI

Anotaciones avanzadas

BigQuery (GCP)
seg_index + ann_index

OncovisionAI — Dataset Explorer