• Uncategorized

About python : S3-upload-from-python

Question Detail

I want to upload folder in S3, but I have lot of folders which have space between their names. It’s an automate process, so could not change the name of the folders manually (from abc d to ‘abc d’).

folder_name = abc d
import os
cmd = 'aws s3 sync'+''+folder_name+'/'+''+s3path+folder_name+'/'
os.system(cmd)

Question Answer

Will be pasting ,my code here. Will be glad if it helps:)

!pip install boto3  ##install boto3
!pip install s3fs  ##required package

import boto3
import pandas as pd

s3 = boto3.client('s3')

s3 = boto3.resource(
service_name='s3',
region_name='us-east-2',
aws_access_key_id='',
aws_secret_access_key='')

# Upload files to S3 bucket
s3.Bucket('bucket_name').upload_file(Filename='foo.csv', Key='foo.csv')
s3.Bucket('bucket_name').upload_file(Filename='bar.csv', Key='bar.csv')

for obj in s3.Bucket('bucket_name').objects.all():
    print(obj)

Note :

  1. the last for loop will print the keys of files that you’ve
    uploaded (for confirmation).
  2. aws_access_key_id and aws_secret_access_key – both of these will be available in
    your aws profile.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.