S3 connection error with Redshift provider

More
5 years 2 months ago #10434 by danielbos
danielbos created the topic: S3 connection error with Redshift provider
Hi all,

I have created a SSIS package with OLE DB connections which inserts data to a simple table. I wanted to test the speed of the PGNP Redshift connector.

Since PGNP also uses the COPY command I have set the S3 information like:

Bulk insert method: Copy command
Bulk insert buffer size: 1000000
AWS Access Key: myaccesskey
AWS Secret Key: mysecretkey
AWS Bucket: s3:\\mybucketname

When I test the package I'll get an error:
[OLE DB Destination [2]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Redshift OLE DB Provider" Hresult: 0x80004005 Description: "ERROR: S3ServiceException:The specified bucket does not exist,Status 404,Error NoSuchBucket

What am I doing wrong? I have set various versions of the AWS Bucket: for example just mybucketname or [url=s3://mybucketname]s3://mybucketname[/url] or s3:\\mybucketname\

Could you help me out?

Daniel

Please Log in or Create an account to join the conversation.

More
5 years 2 months ago #10435 by Moderator
Moderator replied the topic: Re: S3 connection error with Redshift provider
Hi Daniel,
please do not add any prefix to the bucket name, just keep it as "mybucketname".

Also, I do not recommend using bulk insert size of 1000000. Depending on a row size, it could stress RAM. Start with 10000, and see if it satisfies your performance goal.

Best regards,
Moderator

Please Log in or Create an account to join the conversation.

More
5 years 2 months ago #10437 by danielbos
danielbos replied the topic: Re: S3 connection error with Redshift provider
That did the trick!

Thanks!

Please Log in or Create an account to join the conversation.

Time to create page: 0.113 seconds
Powered by Kunena Forum