What to do when a CSV is too big for Excel

You export the table, double click the file, and one of two things happens. Excel refuses, or it opens something that looks fine until you notice the row count stopped at a suspiciously round number.

That number is 1,048,576. It is the maximum number of rows a worksheet can hold. Anything past it does not arrive, and the file gives you no particular warning that it is missing.

The short answer

You need a tool that does not try to put every row in a grid at once, and you need the steps you apply to be repeatable, because you will be asked to run this again next month.

Those are two separate requirements and most quick fixes solve only the first.

Why the quick approaches go wrong

Splitting the file into chunks. Now you can open it. You also cannot count anything across the whole dataset, because no single chunk contains it. Any total, any deduplication, any group by is now wrong in a way that is easy to miss.

Opening it in a text editor. Good for looking at the first few lines and confirming the delimiter. Not a way to compute anything.

Sampling the first 100,000 rows. Fine for a sanity check. Dangerous as an answer, because exports are often sorted, so the first 100,000 rows are frequently the oldest or the smallest or all from one region.

Doing it by hand and hoping to remember. This is the requirement people skip. Six weeks later someone asks why the number moved and the honest answer is that nobody knows what was done to the file.

What to check in any tool you consider

1. Does it work past the grid limit? Not "does it open the file" but "does it compute over all of it". Some tools open a large file by showing you a window into it and quietly compute on that window.

2. Can you replay the work? You want the transformation stored as steps you can run again on next month's export, not as a sequence of clicks you performed once.

3. Where does the data go? If the export contains customer or financial data, a browser tool that uploads it has moved your problem rather than solved it.

The check that belongs to no product

If you are evaluating anything that runs in a browser, open developer tools, go to the Network tab, clear it, then load your file and run a transformation.

If the work is happening somewhere else, a request has to carry your data there and you will see it. Local processing shows requests for the tool's own code and nothing containing your rows. This takes about thirty seconds and it settles the question for any vendor, including us.

Free ways to do it

DuckDB is the strongest free answer here and you should try it first. It queries CSV files directly with SQL, handles files far larger than memory, and costs nothing. If you are comfortable writing a SELECT, this is very likely where you should stop reading.

Python with pandas or polars is the other obvious route, and gives you a reproducible script as a side effect. It needs you to be comfortable in a notebook or a terminal.

Both are free and both are genuinely good. The reason people keep looking is that both ask you to leave the spreadsheet way of working, and a lot of analysts are fast in a grid and slow in SQL.

Where TableRune fits

TableRune is our spreadsheet for datasets that have outgrown one. It keeps the grid interface and runs in the browser tab, on your own machine.

The specific things it does for this problem:

• It handles up to 500,000 rows on the free tier and up to 10 million on the paid one. Both ceilings are enforced inside the engine rather than in the page around it.
• Every transformation is expressed as a pipeline rather than as a set of edits you made once. The pipeline is the single source of truth: changing anything rebuilds it and re-materializes the result, and undo and redo swap which version of it is current. That is what makes next month's run a replay instead of a reconstruction.
• Formulas are parsed by the engine, so you write them the way you already do rather than learning a query language.

It runs DuckDB. We should be plain about this rather than let you find out later. The query engine underneath TableRune is DuckDB-Wasm, bundled with the app so it fetches nothing at runtime, running in a Web Worker. The part we wrote sits on top: the grid, the pipeline, and the translation from what you clicked into what DuckDB executes.

Cost. Free up to 500,000 rows. Paid above that, up to 10 million.

How it differs from the free options. Since it is the same engine, the honest answer is that you are not buying more query power. You are buying the interface to it. If you are comfortable writing SQL against your CSV, use DuckDB directly, and you will have the better tool for the job. If the person who has to run this next month is an analyst who works in a grid, that is the gap this fills. The same reasoning applies to pandas.

The one thing you cannot get from either free option is somebody else maintaining the pipeline format, and that is worth very little if you were happy writing the script.

If you are fluent in SQL, use DuckDB. This is worth a look when the people who need to run and change the work are not, or when the file contains data you are not allowed to send to a hosted tool.