Devart Forums

I'm currently evaluating the SalesForce SSIS Data Flow Components to see if it would be a suitable replacement in our company for our existing solution.

We source millions of rows from SalesForce to a SQL Server database on a daily basis.

I have set up a sample SSIS package to pull from the SalesForce Account table. I have set the property of the Devart SalesForce Source to have BulkQuery = True. However, it doesn't appear like it is actually using the bulk API. I checked the "Monitor Bulk Data Load Jobs" page in SalesForce and its not showing my username there. Is there something else I have to do to get it to use the bulk API?

I also have an additional question. The maximum batch size it will allow me to select is 2000. We will very quickly burn through our 10,000 batches with this setting. Is there a way to increase the batch size to avoid exceeding the bulk API batch limit?

Thanks!

sharon wrote: ↑Tue 29 May 2018 15:05I have set up a sample SSIS package to pull from the SalesForce Account table. I have set the property of the Devart SalesForce Source to have BulkQuery = True. However, it doesn't appear like it is actually using the bulk API. I checked the "Monitor Bulk Data Load Jobs" page in SalesForce and its not showing my username there. Is there something else I have to do to get it to use the bulk API?

Bulk API is designed to quickly load data INTO Salesforce. So, this feature is used by Devart Salesforce Destination.

The BulkQuery property of Devart Salesforce Source determines whether to use optimizations for retrieving large amount of data. If True, provider tries to apply ORDER BY and LIMIT to get parts of result set step by step to avoid possible timeouts. Default value is False.

sharon wrote: ↑Tue 29 May 2018 15:05I also have an additional question. The maximum batch size it will allow me to select is 2000. We will very quickly burn through our 10,000 batches with this setting. Is there a way to increase the batch size to avoid exceeding the bulk API batch limit?

The BatchSize property of Devart Salesforce Source specifies the number of rows to be retrieved as a batch. The maximum is 2000. Default is 500.

Refer to https://www.devart.com/dotconnect/sales ... Calls.html (doc is for our different product but it will help you to understand the way how API calls are countered, the difference is that the "SSIS Data Flow Components" product doesn't use local cache files but caches data in memory).

* The Bulk API functionality is implemented in the Devart Salesforce Source component via its BulkQuery property (Salesforce)
* The OptimizeQuery property is added to the Devart Salesforce Source component to use optimizations for retrieving large amount of data (Salesforce)

For more information, refer to HTML Help documentation (download, right-click the file > Properties > Unblock), navigate to Custom Properties Reference > Devart Salesforce Source.