Amazon S3 PowerShell commands

I recently purchased a QNap NAS to store all of our photos, and I am backing it up to Amazon AWS S3.  I want to be able to manage it via S3 PowerShell, and am finding that the documentation and other available bits and pieces related to managing S3 via PowerShell are lacking.  As such, This is a document outlining what I have found.

 

Getting started

First off, you need to setup your credentials

Set-AWSCredentials -AccessKey AKIAIOSFODNN7EXAMPLE -SecretKey wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY -StoreAs MyProfileName

Next, set the credentials to be the default credentials for this and all future powershell sessions.

Initialize-AWSDefaults -ProfileName MyProfileName -Region us-west-2

Now, I can create a new bucket.  I want to create the bucket via PowerShell. If I create the bucket via the S3 console, I will be unable to fully manage it with PowerShell in the future, so let’s start right.

New-S3Bucket -BucketName mybucketname

Let’s tag the bucket

PS C:\Users\Jeff> Write-S3BucketTagging -BucketName mybucketname -TagSet  @( @{ Key=”Name”; Value=”Qnap Replications” }, @{ Key=”Usage”; Value=”Backups” } )

Get-S3BucketTagging -BucketName mybucketname

Key                                                         Value
—                                                         —–
Name                                                        Qnap Replications
Usage                                                       Backups

Get a count of items in my bucket

(Get-S3Object -BucketName mybucketname | ft Key).count

Get a list of the items in my bucket

Get-S3Object -BucketName mybucketname | ft Key

Place the list of items in a variable to continue to work with it

$Items = Get-S3Object -BucketName mybucketname

List all items in the bucket, smallest to largest

 $Items | Sort-Object size | ft size,LastModified,Key -a

Get only items in a specific subfolder (note, the -KeyPrefix is the folder name, and is case sensitive)

Get-S3Object -BucketName mybucketname -KeyPrefix homes/Jeff

Get a list of all files of a specific file type, or a specific file naming.

Get-S3Object -BucketName mybucketname |Where-Object Key -like “*.png”

Get-S3Object -BucketName mybucketname |Where-Object Key -like “*singapore*”

Configure a Lifecycle management policy that moves all backups to AWS Glacier after 60 days.  This policy does NOT delete any files at any time.

$LifecycleTransition = New-object Amazon.S3.Model.LifecycleTransition
$LifecycleTransition.Days = 60
$LifecycleTransition.StorageClass = ‘Glacier’

$LifecycleRule = New-Object Amazon.S3.Model.LifecycleRule
$LifecycleRule.Id = “Glacier60Days”
$LifecycleRule.Transition = $LifecycleTransition
$LifecycleRule.Prefix = $null

$LifecycleRule.Status = “Enabled”
Write-S3LifecycleConfiguration -BucketName mybucketname -Configuration_Rules $LifecycleRule

If I want to expire (delete) any of this data, I could add the following in my parameters above.  The below will delete anything that has been in S3/Glacier for 365 days.  Since I am using this as a backup of our family’s most precious photos, I will not be expiring any data.

$LifecycleExpiration = New-Object Amazon.S3.Model.LifecycleRuleExpiration
$LifecycleExpiration.Days = 365

$LifecycleRule.Expiration = $LifecycleExpiration

If I want to download an item from S3 via PowerShell, I can do the following:

File: PS C:\Users\Jeff> Read-S3Object -BucketName mybucketname -Key Multimedia/Pictures/2006/IMG_6648.JPG -File C:\Users
\Jeff\Desktop\IMG-6648.jpg

What if I want to download a group of files.  This recently happened.  I downloaded a number of photos from the camera to our Qnap and the next day headed out to the airport for a flight.  My daughter asked me “you did copy those photos to the computer, right?  I need them”.  Well, no I didn’t.  Thankfully, the Qnap uploads files quite rapidly to S3, so the images were in my S3 bucket.  Here is what I did to download all the photos in the required folder, while waiting at the airport gate.

#First, let’s grab all files under the needed folder into a variable

$NancyLPSMV = Get-S3Object -BucketName mybucket | Where-Object Key -like “Multimedia/Pictures/!NancyLPSPhotos/Lps MVs/Lps Meps/Others Meps/How to be a Heartbreaker*”

#Now to download

$NancyLPSMV | %{$Key = $($_.key);Read-S3Object -BucketName
MyBucket -KeyPrefix “Multimedia/Pictures/!NancyLPSPhotos/Lps MVs/Lps Meps/Others Meps/How to be a Heartbreaker” -Folder C:\temp\nancymv\}

Leave a Reply

Your email address will not be published. Required fields are marked *