Data export with large dataset
Posted: Fri 12 Oct 2018 09:05
Hi
When you run a query and select export results, it exports only the current page (so, for example, the first 1000 rows). However, for testing large dataset, I need to export ALL the results.
The only way I found was to disable the pagination, but the issue with this is that when you want to export you have to wait for dbForce to load the ENTIRE result in memory and then export.
Is there a way to run a query and tell it to NOT load in memory but just save to disk as CSV?
Thanks
When you run a query and select export results, it exports only the current page (so, for example, the first 1000 rows). However, for testing large dataset, I need to export ALL the results.
The only way I found was to disable the pagination, but the issue with this is that when you want to export you have to wait for dbForce to load the ENTIRE result in memory and then export.
Is there a way to run a query and tell it to NOT load in memory but just save to disk as CSV?
Thanks