Bulk Insert in Google BigQuery error

Discussion of issues, suggestions and bugs of Devart SSIS Data Flow Components, our product line for building SSIS-based ETL solutions, performing data access to popular cloud applications and databases.
Post Reply
coa
Posts: 4
Joined: Mon 18 Feb 2019 21:07

Bulk Insert in Google BigQuery error

Post by coa » Mon 18 Feb 2019 21:10

You already see this error?

[Devart BigQuery Destination [20]] Error: System.Exception: An exception has occurred during insert operation, the message returned from the provider is:
Value can not be null.
Parameter Name: requestUriString ---> Devart.Data.BigQuery.BigQueryException: Value can not be null.
Parameter Name: requestUriString ---> System.ArgumentNullException: Value can not be null.
Parameter Name: requestUriString
em System.Net.WebRequest.Create(String requestUriString)
em Devart.Data.Rest.b.a(String A_0, String A_1, String A_2, String A_3, Dictionary`2 A_4, Nullable`1 A_5)
em Devart.Data.Rest.c.g7(String A_0, String A_1, Stream A_2, String A_3, Dictionary`2 A_4, h A_5, Nullable`1 A_6, Nullable`1 A_7, Nullable`1 A_8, String A_9)
em Devart.Data.BigQuery.ab.b.a()
em Devart.Data.bd.a[a](Boolean A_0, Int32 A_1, Func`1 A_2)
--- Fim do rastreamento de pilha de exceções internas ---
em Devart.Data.bd.a[a](Boolean A_0, Int32 A_1, Func`1 A_2)
em Devart.Data.BigQuery.ab.a(String A_0, String A_1, Stream A_2)
em Devart.Data.BigQuery.e.a()
em Devart.Data.BigQuery.e.hg()
em Devart.SSIS.BigQuery.BigQueryDestination.ProcessResults(PipelineBuffer buffer, IDTSOutput100 output, IDTSOutput100 errorOutput, Boolean redirect)
em Devart.SSIS.SqlShim.DestinationComponent.ProcessInput(Int32 inputID, PipelineBuffer buffer)
--- Fim do rastreamento de pilha de exceções internas ---
em Devart.SSIS.SqlShim.DestinationComponent.ProcessInput(Int32 inputID, PipelineBuffer buffer)
em Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)

Pinturiccio
Devart Team
Posts: 2406
Joined: Wed 02 Nov 2011 09:44

Re: Bulk Insert in Google BigQuery error

Post by Pinturiccio » Fri 22 Feb 2019 17:50

We have answered you via e-mail.

coa
Posts: 4
Joined: Mon 18 Feb 2019 21:07

Re: Bulk Insert in Google BigQuery error

Post by coa » Fri 22 Feb 2019 18:19

Hi,

We see at a POST that works and the other that does not work and the problem is the field "selfLink", that will be use do do the future command DELETE, and how this fileld does not exists at the respose message, this command will be executed at a null value in the System.Net.WebRequest.Create(String requestUriString) generating the exception "Value can not be null".

The requests are the same, but the response are little differents.

All the responses that works are from "Server: GSE", that does not works are from "Server: ESF".

Pinturiccio
Devart Team
Posts: 2406
Joined: Wed 02 Nov 2011 09:44

Re: Bulk Insert in Google BigQuery error

Post by Pinturiccio » Mon 25 Feb 2019 15:49

We have fixed the "Value can not be null. Parameter Name: requestUriString" error. We will post here when the corresponding build of SSIS Data Flow Components for BigQuery is available for download.

However, we can't tell approximately when it will be available.

The internal build is a full-featured SSIS Data Flow Components build, and can be used in the same way as the public build. You can download the internal build of SSIS Data Flow Components with this change from our site http://devart.com/pub/devartssis1_10_892.exe

The internal build is a full-featured SSIS Data Flow Components build and can be used in the same way as a public build.

Please post here about the results of using this internal build.

coa
Posts: 4
Joined: Mon 18 Feb 2019 21:07

Re: Bulk Insert in Google BigQuery error

Post by coa » Thu 28 Feb 2019 15:54

This component is generating another error.

Maybe it does not reading the bucket name property.

System.Exception: An exception has occurred during insert operation, the message returned from the provider is:
userxxx does not have storage.buckets.get access to default. ---> Devart.Data.BigQuery.BigQueryException: userxxx does not have storage.buckets.get access to default.
at Devart.Data.bd.a[a](Boolean A_0, Int32 A_1, Func`1 A_2)
at Devart.Data.BigQuery.ab.a(String A_0)
at Devart.Data.BigQuery.e.g9()
at Devart.SSIS.SqlShim.DestinationComponent.ProcessInput(Int32 inputID, PipelineBuffer buffer)
--- End of inner exception stack trace ---
at Devart.SSIS.SqlShim.DestinationComponent.ProcessInput(Int32 inputID, PipelineBuffer buffer)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)

Pinturiccio
Devart Team
Posts: 2406
Joined: Wed 02 Nov 2011 09:44

Re: Bulk Insert in Google BigQuery error

Post by Pinturiccio » Thu 07 Mar 2019 14:59

coa wrote:userxxx does not have storage.buckets.get access to default.
This issue occurs when using the default bucket because of BigQuery limitations. This bucket doesn't suit for operations involving BigQuery API.

Please try using another bucket in your connection.

coa
Posts: 4
Joined: Mon 18 Feb 2019 21:07

Re: Bulk Insert in Google BigQuery error

Post by coa » Thu 21 Mar 2019 21:06

The root cause of the problem was fixed by Google team.

https://issuetracker.google.com/issues/128847197

Post Reply