SSIS Salesforce component not batching properly

Discussion of issues, suggestions and bugs of Devart SSIS Data Flow Components, our product line for building SSIS-based ETL solutions, performing data access to popular cloud applications and databases.
Post Reply
ibcrandy
Posts: 1
Joined: Fri 11 Aug 2017 19:02

SSIS Salesforce component not batching properly

Post by ibcrandy » Fri 11 Aug 2017 19:10

Hi -

So I have a medium sized number of objects I need to load to salesforce (about 13 objects ranging from 10K to 60K records each). Some of my objects seem to work correctly and batch in high numbers, while others are spitting out batches of 2. The latter is an issue since I only have a 10K per day batch call limit. Support said that the objects are too large and that's what's causing the problem, but after trimming them down a little I'm still hitting the issue even though other objects are much larger and batching well. For some concrete numbers:

The item in question has 55K records and should be about 350 bytes each, for a total of ~16.5M bytes.

An item that loads well only has 27K records but they are ~1500 bytes each, for a total of ~40M bytes.

There is only one package in my solution and all components are set with the same batch and cache sizes, so I can't figure out why one will batch with ~2700 records per batch while the other does exactly 2 every batch. If anyone has any suggestions I am open to trying anything at this point. Thanks!

Pinturiccio
Devart Team
Posts: 2420
Joined: Wed 02 Nov 2011 09:44

Re: SSIS Salesforce component not batching properly

Post by Pinturiccio » Tue 15 Aug 2017 15:04

We have answered you via e-mail.

Post Reply