aws

AWS CloudFormation · March 28, 2019 · amazon aws

If you’re doing any production-level work in AWS, you should be using AWS CloudFormation. It’s really easy to get started. Let’s walk through the basics. Why use CloudFormation? Here’s a common scenario: creating an EC2 instance and assigning an Elastic IP address.…

Easily Finding your AWS S3 Canonical ID · February 28, 2019 · aws s3

Another quick post – found this in the AWS Console UI. If you ever need to share your AWS Canonical ID with someone, e.g. to share S3 buckets. You can find your AWS Canonical ID by using various APIs – but I was also able to find it using…

AWS Certified Solutions Architect · August 8, 2018 · amazon aws ec2 linux

Quick post – I’ve been busy studying for the AWS Certified Solutions Architect – Associate exam for the past few weeks – good news, I passed it a few days ago! Shoot me a note if you ever need some solutions architected. I primarily did this because I&…

azssh: Easily manage EC2 instances · June 16, 2018 · aws go golang

azssh is a small commandline utility I wrote a few months ago to help with managing EC2 instances. My workflow on EC2 consists of starting and stopping instances and sometimes SSHing in to run some commands. That’s what this utility does – starts and stops EC2 instances, tells…

Jupyter Notebooks with PySpark on AWS EMR · October 16, 2017 · aws emr jupyter pyspark python spark

One of the biggest, most time-consuming parts of data science is analysis and experimentation. One of the most popular tools to do so in a graphical, interactive environment is Jupyter. Combining Jupyter with Apache Spark (through PySpark) merges two extremely powerful tools. AWS EMR lets you set up all of…

Spark and Kinesis Optimization · January 29, 2017 · aws kinesis python spark streaming

You’ll definitely want to read this if you’re using AWS Kinesis with Apache Spark to stream data, it’s been extremely valuable: https://aws.amazon.com/blogs/big-data/optimize-spark-streaming-to-efficiently-process-amazon-kinesis-streams/…

EC2 + Route53 for Dynamic DNS · March 12, 2016 · aws ec2 go golang

Recently I ran into a problem while working with Amazon EC2 servers. Servers without dedicated elastic IP addresses would get a different IP address every time they were started up! This proved to be a challenge when trying to SSH in to the servers. How can I have a dynamic…