Python ETL (lxml/Pandas).
Stream the gigabyte, never load it. Iterparse plus a pre-processor beats RAM every time.
Supplier feeds arrive as gigabyte XML nobody validated. A pre-processor normalizes the dialect, then lxml.etree.iterparse streams elements while Pandas handles the aggregation frame. Constant memory regardless of feed size.
The pipeline pattern behind ~/work/inventory-forecasting-tool.
The canonical input was a 1.8 GB Tally XML export the supply-chain team used to wrangle through crashing Excel sheets. The run log shows parsing the full dump, then fitting Prophet per SKU with 8 workers until all 7,800 models converged. Stockouts and tied-up capital became a CSV replenishment plan. See ~/work/inventory-forecasting-tool.
used_in: ~/work/inventory-forecasting-tool