S3 uploading from Stdin using AWS Go SDK

This post is about the correct way to use os.Stdin for S3 uploading using AWSK Go SDK.

Recently I was writing a CLI app that is supposed to read data from stdin and upload it into S3 using AWS Go SDK. The implementation was straightforward but the app terminated with the error seek /dev/stdin: illegal seek.

The usage of stdin with S3 SDK is something similiar to below snippet

    uploadInput := &s3.PutObjectInput{
        Bucket: bucket,
        Key:    key,
        Body:   os.Stdin,
    }

Further investigation revealed that AWS Go SDK internally checks the type of Body and it tries to determine the length of the body by seeking if the type supports the Seek method.

Incidentally, os.Stdin is of type os.File which provides Seek method(eventhough it fails with an error). So AWS SDK tries to determine the length of Body which triggers the error I mentioned above.

One way we can avoid the length determination(and the error) is by hiding the fact that os.Stdin the Seek method. We can do this by simply wrapping os.Stdin with io.MultiReader which provides Read method only.

So I got the app working by changing the stdin usage like below

    uploadInput := &s3.PutObjectInput{
        Bucket: bucket,
        Key:    key,
        Body:   io.MultiReader(os.Stdin),
    }

Checkout this link for Robpike proposal to change io.StdXXX in Go 2. Thanks to https://twitter.com/sudproquo for the link.