
I am a huge fan of [AWS Lambda](https://aws.amazon.com/lambda/) and serverless in general. I also love the reliability, speed, and simplicity of [SQLite](https://www.sqlite.org), and I was looking to find a few projects to use it.

The issue was that until recently you could not use AWS Lambda together with SQLite, other than bundling the whole database file into the Lambda bundle zip file itself. In practice this meant you only had a readonly database though, since each concurrent Lambda invocation would have its own copy of the database.

However, last month, AWS released [integration between AWS Lambda and Amazon EFS](https://aws.amazon.com/blogs/compute/using-amazon-efs-for-aws-lambda-in-your-serverless-applications/). Since Amazon EFS already supported [NFSv4 lock upgrading/downgrading](https://aws.amazon.com/about-aws/whats-new/2017/03/amazon-elastic-file-system-amazon-efs-now-supports-nfsv4-lock-upgrading-and-downgrading/) which is needed by SQLite, it means that now we can access an SQLite database file stored on an EFS filesystem through AWS Lambda in a read-write mode 🔥 🚀

## Quick Test

I did a quick test using a Node.js Lambda function and it seems that the EFS connectivity adds around **100ms** of latency overhead to the Lambda invocations when accessing the database over EFS versus a local filesystem. This test is by no means scientific and performance will vary a lot depending on your database file size, the concurrent connections you have, and the number of concurrent writes to the database.

The [simple test application (code available)](https://github.com/lambrospetrou/code-playground/blob/master/aws-lambda-node-sqlite/local.js) executes 100 `insert` statements using a transaction, a `select` query with a `where` condition, and then a `select *` query.

**AWS Lambda**

![sqlite-aws-lambda-efs](/articles/aws-lambda-and-sqlite-over-efs/sqlite-lambda-3gb.jpg)

The execution time of the statements accessing the SQLite database is roughly **140ms**. There is some variation but that's what I was averaging after 10+ runs. I also tried several AWS Lambda memory configurations to see if the more powerful CPU would affect the latency in any way but it didn't. This makes me think that the overhead is pure network IO.

**Local - Surface Pro**

![sqlite-local](/articles/aws-lambda-and-sqlite-over-efs/sqlite-local.jpg)

The execution time of the statements accessing the SQLite database is roughly **30-40ms**. Some variation expected here as well albeit lower since it's a local filesystem.

## Is this Production ready?

Most probably not...

The correct answer is **it depends!**

- Accessing an EFS filesystem adds latency to your Lambda invocations just due to the network overhead.
- SQLite was designed to be the best in-process database in the world. SQLite's website itself [describes amazingly well the appropriate uses for SQLite](https://www.sqlite.org/whentouse.html), including the inappropriate ones. Our experiment, since it introduces a network between the application (Lambda code) and the database file (EFS) falls into the inappropriate uses...
- SQLite supports concurrent readers, but only one writer can be writing to the database, therefore only one Lambda invocation will be able to write to the database at any point in time which is going to be a throughput bottleneck.

The above facts make it obvious that using SQLite with AWS Lambda is a very bad idea for production systems with thousands of users, or high-throughput of writes, or any system that needs low latency for that matter.

However, for side projects, small team internal projects, applications where latency is not an issue, or situations when you want to build a quick demo prototype it's probably alright.

I am sure the AWS Lambda users will find **interesting** ways to abuse this integration, but that's expected, and it's half the fun 😜


I am a huge fan of <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> and serverless in general. I also love the reliability, speed, and simplicity of <a href="https://www.sqlite.org">SQLite</a>, and I was looking to find a few projects to use it.
The issue was that until recently you could not use AWS Lambda together with SQLite, other than bundling the whole database file into the Lambda bundle zip file itself. In practice this meant you only had a readonly database though, since each concurrent Lambda invocation would have its own copy of the database.
However, last month, AWS released <a href="https://aws.amazon.com/blogs/compute/using-amazon-efs-for-aws-lambda-in-your-serverless-applications/">integration between AWS Lambda and Amazon EFS</a>. Since Amazon EFS already supported <a href="https://aws.amazon.com/about-aws/whats-new/2017/03/amazon-elastic-file-system-amazon-efs-now-supports-nfsv4-lock-upgrading-and-downgrading/">NFSv4 lock upgrading/downgrading</a> which is needed by SQLite, it means that now we can access an SQLite database file stored on an EFS filesystem through AWS Lambda in a read-write mode 🔥 🚀
<h2 id="quick-test">Quick Test</h2>
I did a quick test using a Node.js Lambda function and it seems that the EFS connectivity adds around 100ms of latency overhead to the Lambda invocations when accessing the database over EFS versus a local filesystem. This test is by no means scientific and performance will vary a lot depending on your database file size, the concurrent connections you have, and the number of concurrent writes to the database.
The <a href="https://github.com/lambrospetrou/code-playground/blob/master/aws-lambda-node-sqlite/local.js">simple test application (code available)</a> executes 100 <code>insert</code> statements using a transaction, a <code>select</code> query with a <code>where</code> condition, and then a <code>select *</code> query.
AWS Lambda
<img src="/articles/aws-lambda-and-sqlite-over-efs/sqlite-lambda-3gb.jpg" alt="sqlite-aws-lambda-efs"/>
The execution time of the statements accessing the SQLite database is roughly 140ms. There is some variation but that’s what I was averaging after 10+ runs. I also tried several AWS Lambda memory configurations to see if the more powerful CPU would affect the latency in any way but it didn’t. This makes me think that the overhead is pure network IO.
Local - Surface Pro
<img src="/articles/aws-lambda-and-sqlite-over-efs/sqlite-local.jpg" alt="sqlite-local"/>
The execution time of the statements accessing the SQLite database is roughly 30-40ms. Some variation expected here as well albeit lower since it’s a local filesystem.
<h2 id="is-this-production-ready">Is this Production ready?</h2>
Most probably not…
The correct answer is it depends!
<ul>
<li>Accessing an EFS filesystem adds latency to your Lambda invocations just due to the network overhead.</li>
<li>SQLite was designed to be the best in-process database in the world. SQLite’s website itself <a href="https://www.sqlite.org/whentouse.html">describes amazingly well the appropriate uses for SQLite</a>, including the inappropriate ones. Our experiment, since it introduces a network between the application (Lambda code) and the database file (EFS) falls into the inappropriate uses…</li>
<li>SQLite supports concurrent readers, but only one writer can be writing to the database, therefore only one Lambda invocation will be able to write to the database at any point in time which is going to be a throughput bottleneck.</li>
</ul>
The above facts make it obvious that using SQLite with AWS Lambda is a very bad idea for production systems with thousands of users, or high-throughput of writes, or any system that needs low latency for that matter.
However, for side projects, small team internal projects, applications where latency is not an issue, or situations when you want to build a quick demo prototype it’s probably alright.
I am sure the AWS Lambda users will find interesting ways to abuse this integration, but that’s expected, and it’s half the fun 😜

Use the amazing SQLite3 database with AWS Lambda over an EFS filesystem.

AWS Lambda and SQLite3 over Amazon EFS


>The OTP team is respecting the old slogan: first make it work, then make it beautiful, and only if you need to, make it fast.

I was reading the [Learn You Some Erlang](https://learnyousomeerlang.com/maps) book this morning and at some point I stumbled upon the above sentence. I heard this phrase online for the first time about 2 years ago from Jose Valim (creator of [Elixir](https://elixir-lang.org/)) during one of his [Advent of Code videos](https://www.twitch.tv/josevalim/videos).

This phrase really resonates with me because this is exactly how I approach writing code and solving problems as well. Let's elaborate a bit more...

## Make it work

The main reason we write code (other than fun) is to solve a problem, and if our solution is not correct, then we did't do our job. **Correctness comes first!**

During this step, I usually write code without too much thinking into abstractions, how to design the components, function names, or other appealing details. I just want to solve the problem.

## Make it beautiful

This step can be intepreted in many ways depending on the programming language used, the domain of the problem, your team, etc. 

However, it could be refactoring the public API of your library/module, coming up with the right abstractions, splitting long functions into smaller more testable ones, and anything else that makes the code readable, extensible, and ready to be reviewed by your peers.

## Make it fast

This step can be considered optional in a lot of day-to-day cases. However, it's always good practice, and over time it can become a habit to check your code for performance issues. 

You solved the problem correctly (step 1), and the code is all pretty and ready to be reviewed (step 2), but is it fast enough? An even better question would be: _Is it slow enough to cause problems?_

The time and effort you put into making your solution fast solely depends on your use-case, but always remember to ask this question.

## Conclusion

Always follow this approach when writing code:
1. Make it work
2. Make it beautiful
3. Make it fast


<blockquote>
The OTP team is respecting the old slogan: first make it work, then make it beautiful, and only if you need to, make it fast.
</blockquote>
I was reading the <a href="https://learnyousomeerlang.com/maps">Learn You Some Erlang</a> book this morning and at some point I stumbled upon the above sentence. I heard this phrase online for the first time about 2 years ago from Jose Valim (creator of <a href="https://elixir-lang.org/">Elixir</a>) during one of his <a href="https://www.twitch.tv/josevalim/videos">Advent of Code videos</a>.
This phrase really resonates with me because this is exactly how I approach writing code and solving problems as well. Let’s elaborate a bit more…
<h2 id="make-it-work">Make it work</h2>
The main reason we write code (other than fun) is to solve a problem, and if our solution is not correct, then we did’t do our job. Correctness comes first!
During this step, I usually write code without too much thinking into abstractions, how to design the components, function names, or other appealing details. I just want to solve the problem.
<h2 id="make-it-beautiful">Make it beautiful</h2>
This step can be intepreted in many ways depending on the programming language used, the domain of the problem, your team, etc. 
However, it could be refactoring the public API of your library/module, coming up with the right abstractions, splitting long functions into smaller more testable ones, and anything else that makes the code readable, extensible, and ready to be reviewed by your peers.
<h2 id="make-it-fast">Make it fast</h2>
This step can be considered optional in a lot of day-to-day cases. However, it’s always good practice, and over time it can become a habit to check your code for performance issues. 
You solved the problem correctly (step 1), and the code is all pretty and ready to be reviewed (step 2), but is it fast enough? An even better question would be: Is it slow enough to cause problems?
The time and effort you put into making your solution fast solely depends on your use-case, but always remember to ask this question.
<h2 id="conclusion">Conclusion</h2>
Always follow this approach when writing code:
<ol>
<li>Make it work</li>
<li>Make it beautiful</li>
<li>Make it fast</li>
</ol>

How I approach writing code and solving problems.

Make it Work, Make it Beautiful, Make it Fast


I have been doing a lot of web development over the past few years, both in my own time with my side projects but also at work. One of the things that you need at some point is to write a script to automate some kind of action usually done in the browser. For example, this could be navigating to certain pages and taking screenshots, or automated form submission, or automated UI testing.

One of the simplest use-cases I have for my own website is to generate a PDF version of [my CV page](https://www.lambrospetrou.com/cv/). There are several reasons why a PDF file could be better sometimes than a website, so it's useful to have one in hand. Since I already had the HTML version, what I used to do for several years was to use Chrome's **Print to PDF** functionality.

However, a couple of years ago, Google released [Puppeteer](https://developers.google.com/web/tools/puppeteer) which allows someone to script a headless version of Chromium. From the second I saw it I knew it was going to be big! Easy, simple and headless browser automation through Node. The potential use-cases are infinite 🛸

Since then it has been used by companies to make amazing things.

- Automated robust UI tests inside headless Chrome, instead of the finicky Webdriver approaches that were the standard back then.
- Performance profiling and benchmarking of web applications.
- Scripting of manual tasks that people were doing like form submission.
- Web crawling through a normal browser to avoid issues with previous approaches that were using browser-like fakes.
- Server-side rendering of dynamic single page JavaScript application to avoid SEO issues.
- Many other things that required an actual browser running a website.

One of the problems with Puppeteer is that it only supports Chromium so far.

I was extremely happy to see a few days ago that Microsoft finally released the v1 version of [Playwright](https://playwright.dev/). Playwright is extremely similar to Puppeteer, but it extends its amazing feature-set by supporting all the three major browsers, Chromium, Firefox, and Webkit 😍 Not only that, but they added very cool features like auto-waiting for element selection which greatly simplifies DOM manipulation.

There are a few features that are not supported by all the browsers, so keep that in mind if something fails in your scripts, make sure that the browser you use supports it.

## Code

From the little time I spent with Playwright its API seems extremely similar to Puppeteer's so it's easy for someone to jump between the two or migrate from one to the other.

As an example, see below an excerpt of the code I use to generate the PDF version of my website.

The full code can be found at [my CV repository](https://github.com/lambrospetrou/lpcv/blob/master/html/build-tool/generate-pdf.js).

### Local server

While I am making changes to my CV I want to be able to generate the PDF and test it without having to push my changes to the live website. The following code starts a local server at port `12345` that serves the assets from inside the `build/` directory.

```javascript
const handler = require('serve-handler');
const http = require('http');
const path = require('path');
const WS_BUILD=path.join(__dirname, './build');

const server = http.createServer((request, response) => {
    return handler(request, response, {
        public: WS_BUILD
    });
})
server.listen(12345)
```

### Puppeteer

The following code is what we need to make Puppeteer visit our local server, wait for the page to fully load, and then generate a PDF named `cv.pdf`.

```javascript
const puppeteer = require('puppeteer');

async function generatePdfPuppeteer() {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('http://127.0.0.1:12345', {waitUntil: 'networkidle2'});
    await page.pdf({
        path: path.join(WS_BUILD, 'cv.pdf'), 
        format: 'A4',
        margin: {
            top: '0.39in',
            left: '0.39in',
            right: '0.38in',
            bottom: '0.38in'
        }
    });
    await browser.close();
}
```

### Playwright

The code for using Playwright is (almost) identical to Puppeteer in our simple use-case.

```javascript
const { chromium } = require('playwright');

async function generatePdfPlaywright() {
    const browser = await chromium.launch();
    const ctx = await browser.newContext();
    const page = await ctx.newPage();
    await page.goto('http://127.0.0.1:12345', {waitUntil: 'networkidle'});
    await page.pdf({
        path: path.join(WS_BUILD, `cv-playwright.pdf`), 
        format: 'A4',
        margin: {
            top: '0.39in',
            left: '0.39in',
            right: '0.38in',
            bottom: '0.38in'
        }
    });
    await browser.close();
}
```

### Issues with WSL on Windows 10

I am using the [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/about) version 1, and I had issues while running both Puppeteer and PlayWright due to some incompatibilities with the browser binaries.

You can find more information for this issue in these discussions:
- https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md#setting-up-chrome-linux-sandbox
- https://github.com/loteoo/hyperstatic/pull/20/files 

To make Chromium work the arguments `--no-sandbox`, `--disable-setuid-sandbox`, and `--single-process` have to be passed to the `launch()` method. See the modified lines below.

#### Puppeteer

```javascript
const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox', '--single-process']});
```

#### Playwright

```javascript
const browser = await chromium.launch({args: ['--no-sandbox', '--disable-setuid-sandbox', '--single-process']});
```

## Conclusion

Having the ability to write a few simple lines of code and automate a full-blown browser is amazing. What I discussed above only scratches the surface of what can be done and I am always excited to see how people use these stuff.

Enjoy 😉



I have been doing a lot of web development over the past few years, both in my own time with my side projects but also at work. One of the things that you need at some point is to write a script to automate some kind of action usually done in the browser. For example, this could be navigating to certain pages and taking screenshots, or automated form submission, or automated UI testing.
One of the simplest use-cases I have for my own website is to generate a PDF version of <a href="https://www.lambrospetrou.com/cv/">my CV page</a>. There are several reasons why a PDF file could be better sometimes than a website, so it’s useful to have one in hand. Since I already had the HTML version, what I used to do for several years was to use Chrome’s Print to PDF functionality.
However, a couple of years ago, Google released <a href="https://developers.google.com/web/tools/puppeteer">Puppeteer</a> which allows someone to script a headless version of Chromium. From the second I saw it I knew it was going to be big! Easy, simple and headless browser automation through Node. The potential use-cases are infinite 🛸
Since then it has been used by companies to make amazing things.
<ul>
<li>Automated robust UI tests inside headless Chrome, instead of the finicky Webdriver approaches that were the standard back then.</li>
<li>Performance profiling and benchmarking of web applications.</li>
<li>Scripting of manual tasks that people were doing like form submission.</li>
<li>Web crawling through a normal browser to avoid issues with previous approaches that were using browser-like fakes.</li>
<li>Server-side rendering of dynamic single page JavaScript application to avoid SEO issues.</li>
<li>Many other things that required an actual browser running a website.</li>
</ul>
One of the problems with Puppeteer is that it only supports Chromium so far.
I was extremely happy to see a few days ago that Microsoft finally released the v1 version of <a href="https://playwright.dev/">Playwright</a>. Playwright is extremely similar to Puppeteer, but it extends its amazing feature-set by supporting all the three major browsers, Chromium, Firefox, and Webkit 😍 Not only that, but they added very cool features like auto-waiting for element selection which greatly simplifies DOM manipulation.
There are a few features that are not supported by all the browsers, so keep that in mind if something fails in your scripts, make sure that the browser you use supports it.
<h2 id="code">Code</h2>
From the little time I spent with Playwright its API seems extremely similar to Puppeteer’s so it’s easy for someone to jump between the two or migrate from one to the other.
As an example, see below an excerpt of the code I use to generate the PDF version of my website.
The full code can be found at <a href="https://github.com/lambrospetrou/lpcv/blob/master/html/build-tool/generate-pdf.js">my CV repository</a>.
<h3 id="local-server">Local server</h3>
While I am making changes to my CV I want to be able to generate the PDF and test it without having to push my changes to the live website. The following code starts a local server at port <code>12345</code> that serves the assets from inside the <code>build/</code> directory.
<pre><code class="language-javascript">const handler = require(&#39;serve-handler&#39;);
const http = require(&#39;http&#39;);
const path = require(&#39;path&#39;);
const WS_BUILD=path.join(__dirname, &#39;./build&#39;);

const server = http.createServer((request, response) =&gt; {
 return handler(request, response, {
 public: WS_BUILD
 });
})
server.listen(12345)</code></pre>
<h3 id="puppeteer">Puppeteer</h3>
The following code is what we need to make Puppeteer visit our local server, wait for the page to fully load, and then generate a PDF named <code>cv.pdf</code>.
<pre><code class="language-javascript">const puppeteer = require(&#39;puppeteer&#39;);

async function generatePdfPuppeteer() {
 const browser = await puppeteer.launch();
 const page = await browser.newPage();
 await page.goto(&#39;http://127.0.0.1:12345&#39;, {waitUntil: &#39;networkidle2&#39;});
 await page.pdf({
 path: path.join(WS_BUILD, &#39;cv.pdf&#39;), 
 format: &#39;A4&#39;,
 margin: {
 top: &#39;0.39in&#39;,
 left: &#39;0.39in&#39;,
 right: &#39;0.38in&#39;,
 bottom: &#39;0.38in&#39;
 }
 });
 await browser.close();
}</code></pre>
<h3 id="playwright">Playwright</h3>
The code for using Playwright is (almost) identical to Puppeteer in our simple use-case.
<pre><code class="language-javascript">const { chromium } = require(&#39;playwright&#39;);

async function generatePdfPlaywright() {
 const browser = await chromium.launch();
 const ctx = await browser.newContext();
 const page = await ctx.newPage();
 await page.goto(&#39;http://127.0.0.1:12345&#39;, {waitUntil: &#39;networkidle&#39;});
 await page.pdf({
 path: path.join(WS_BUILD, `cv-playwright.pdf`), 
 format: &#39;A4&#39;,
 margin: {
 top: &#39;0.39in&#39;,
 left: &#39;0.39in&#39;,
 right: &#39;0.38in&#39;,
 bottom: &#39;0.38in&#39;
 }
 });
 await browser.close();
}</code></pre>
<h3 id="issues-with-wsl-on-windows-10">Issues with WSL on Windows 10</h3>
I am using the <a href="https://docs.microsoft.com/en-us/windows/wsl/about">Windows Subsystem for Linux (WSL)</a> version 1, and I had issues while running both Puppeteer and PlayWright due to some incompatibilities with the browser binaries.
You can find more information for this issue in these discussions:
<ul>
<li><a href="https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md#setting-up-chrome-linux-sandbox">https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md#setting-up-chrome-linux-sandbox</a></li>
<li><a href="https://github.com/loteoo/hyperstatic/pull/20/files">https://github.com/loteoo/hyperstatic/pull/20/files</a> </li>
</ul>
To make Chromium work the arguments <code>--no-sandbox</code>, <code>--disable-setuid-sandbox</code>, and <code>--single-process</code> have to be passed to the <code>launch()</code> method. See the modified lines below.
<h4 id="puppeteer-1">Puppeteer</h4>
<pre><code class="language-javascript">const browser = await puppeteer.launch({args: [&#39;--no-sandbox&#39;, &#39;--disable-setuid-sandbox&#39;, &#39;--single-process&#39;]});</code></pre>
<h4 id="playwright-1">Playwright</h4>
<pre><code class="language-javascript">const browser = await chromium.launch({args: [&#39;--no-sandbox&#39;, &#39;--disable-setuid-sandbox&#39;, &#39;--single-process&#39;]});</code></pre>
<h2 id="conclusion">Conclusion</h2>
Having the ability to write a few simple lines of code and automate a full-blown browser is amazing. What I discussed above only scratches the surface of what can be done and I am always excited to see how people use these stuff.
Enjoy 😉

I very briefly go through two of the most promising browser automation frameworks, Google's Puppeteer and Microsoft's Playwright.

Enjoyable browser automation with Puppeteer and Playwright

Last week I wrote about [note taking being my productivity tool](https://www.lambrospetrou.com/articles/digital-braindump-and-productivity-tools/), and then about [the worklog](https://www.lambrospetrou.com/articles/best-tip-the-worklog/) being my most important daily document.

Since then, I had several friends and colleagues reaching out asking specifics about the worklog's format and how I structure its content. I will try to briefly describe what I use hoping that it can help others as well.

Before we dive in, I would like to reiterate once again that I don't really believe in the **one size fits all** idea, and what works for me might not work for you. However, seeing the structure I use might inspire you to find one that works for you.

## What?

First of all, I urge you to read [the original article explaining what the worklog is](https://www.lambrospetrou.com/articles/best-tip-the-worklog/) to better understand the rest of this article.

The worklog contains essentially three things:

1. Meeting notes
 - Who was in the meeting
 - When was the meeting
 - What was said during the meeting
2. Activity log
 - What did I do in each day
3. Todo items 
 - Tasks for the next 1-3 weeks (quite fine-grained)
 - Couple of long-term things that I need to break down

## How?

I use the following format for my worklog file:

```markdown
# Worklog 2020-01-01 to present

## Meetings

### 1:1 with manager <userxxx@abc.com> @ 2020-05-10

- topic 1
- topic 2
- action item A
- action item B
- ...

### Meeting for feature XXX @ 2020-04-29

- Who
 + nameA <usera@abc.com>
 + nameB <userb@abc.com>
- thing 1
- thing 2
- action item A
- agreements
 + a
 + b
- open questions...

## Activity log

- Meeting for XXX (2020-04-29)
- Worked on feature Y (2020-04-30)
 + <Task URL> 
 + <Code review URL> implements ZZZ
- Watched training videos
 + infra (2020-05-02)
 + networking (2020-05-03)
- Investigated YYY (2020-05-03)
 + <link to the file containing details or to the internal issue>

## TODO

- implement DDD
 + <internal issue link>
- training videos
 + <link>
- do a deep dive on XXX
```

As you see, it's quite simple, three separate ordered lists.

You will notice that the meetings section orders the meetings by date in descending order (latest at the top), whereas the activity log section orders items by date in ascending order (latest at the bottom). This is super handy for me because over time the file grows, and in my productive months it can get hundreds of lines—each worklog file spans a period of 6 months. This structure allows me to instantly scroll to the top and start writing about the current meeting, or scroll to the bottom and start writing about today's activities, without wasting time figuring out where I need to write. Almost all the text editors and IDEs have shortcuts to move at the top and bottom of a file which makes this approach efficient.

Regarding the todo section, I try to keep it short and not have more than 10 items, since at that point it becomes a backlog. I want items that I will work on during the next 2-3 weeks ordered in priority, with top being the next one to be done. Having said that, I almost always find myself having 1-2 items that are long-term oriented, for which I need to do some kind of investigation. It's useful to have them here to remind myself to do a bit of work every day.

Every morning, I open the worklog, remind myself what needs to be done by going over the todo section, and I make sure it is synced with my team's backlog in case priorities shifted.

That's it folks, I hope this is helpful!

Last week I wrote about <a href="https://www.lambrospetrou.com/articles/digital-braindump-and-productivity-tools/">note taking being my productivity tool</a>, and then about <a href="https://www.lambrospetrou.com/articles/best-tip-the-worklog/">the worklog</a> being my most important daily document.
Since then, I had several friends and colleagues reaching out asking specifics about the worklog’s format and how I structure its content. I will try to briefly describe what I use hoping that it can help others as well.
Before we dive in, I would like to reiterate once again that I don’t really believe in the one size fits all idea, and what works for me might not work for you. However, seeing the structure I use might inspire you to find one that works for you.
<h2 id="what">What?</h2>
First of all, I urge you to read <a href="https://www.lambrospetrou.com/articles/best-tip-the-worklog/">the original article explaining what the worklog is</a> to better understand the rest of this article.
The worklog contains essentially three things:
<ol>
<li>Meeting notes<ul>
<li>Who was in the meeting</li>
<li>When was the meeting</li>
<li>What was said during the meeting</li>
</ul>
</li>
<li>Activity log<ul>
<li>What did I do in each day</li>
</ul>
</li>
<li>Todo items <ul>
<li>Tasks for the next 1-3 weeks (quite fine-grained)</li>
<li>Couple of long-term things that I need to break down</li>
</ul>
</li>
</ol>
<h2 id="how">How?</h2>
I use the following format for my worklog file:
<pre><code class="language-markdown"># Worklog 2020-01-01 to present

## Meetings

### 1:1 with manager &lt;userxxx@abc.com&gt; @ 2020-05-10

- topic 1
- topic 2
- action item A
- action item B
- ...

### Meeting for feature XXX @ 2020-04-29

- Who
 + nameA &lt;usera@abc.com&gt;
 + nameB &lt;userb@abc.com&gt;
- thing 1
- thing 2
- action item A
- agreements
 + a
 + b
- open questions...

## Activity log

- Meeting for XXX (2020-04-29)
- Worked on feature Y (2020-04-30)
 + &lt;Task URL&gt; 
 + &lt;Code review URL&gt; implements ZZZ
- Watched training videos
 + infra (2020-05-02)
 + networking (2020-05-03)
- Investigated YYY (2020-05-03)
 + &lt;link to the file containing details or to the internal issue&gt;

## TODO

- implement DDD
 + &lt;internal issue link&gt;
- training videos
 + &lt;link&gt;
- do a deep dive on XXX</code></pre>
As you see, it’s quite simple, three separate ordered lists.
You will notice that the meetings section orders the meetings by date in descending order (latest at the top), whereas the activity log section orders items by date in ascending order (latest at the bottom). This is super handy for me because over time the file grows, and in my productive months it can get hundreds of lines—each worklog file spans a period of 6 months. This structure allows me to instantly scroll to the top and start writing about the current meeting, or scroll to the bottom and start writing about today’s activities, without wasting time figuring out where I need to write. Almost all the text editors and IDEs have shortcuts to move at the top and bottom of a file which makes this approach efficient.
Regarding the todo section, I try to keep it short and not have more than 10 items, since at that point it becomes a backlog. I want items that I will work on during the next 2-3 weeks ordered in priority, with top being the next one to be done. Having said that, I almost always find myself having 1-2 items that are long-term oriented, for which I need to do some kind of investigation. It’s useful to have them here to remind myself to do a bit of work every day.
Every morning, I open the worklog, remind myself what needs to be done by going over the todo section, and I make sure it is synced with my team’s backlog in case priorities shifted.
That’s it folks, I hope this is helpful!

I briefly describe the structure of my worklog file and why it works for me.

The worklog format 1.0


In a previous article where I wrote about [my favourite productivity tool](https://www.lambrospetrou.com/articles/digital-braindump-and-productivity-tools/) I briefly mentioned about my **daily work log** habbit. This article is its origin story.

## Origin

About 5 years ago I returned to Amazon (did an internship the year before) as a full-time employee after finishing my Masters degree. During one of my first 1:1 meetings with my manager we were talking about career growth and to this day I remember him saying the following.

>If I can only give you one tip, it's to keep notes of what you do every day. Not for the usual reasons like performance reviews, but because there will be days that you will feel unproductive, thinking that you don't produce anything, and it will make you feel bad. Those notes will be the only way to go back and see all the work you did during those days.

This is by far the best tip I received in my (albeit short) career, and it's the first tip I give to every new hire joining my team as well.

I call these notes **the worklog**, and it's essentially a big text file with everything I do throughout the day in chronological order. Anything can go into the worklog like tasks or issues I worked on, code reviews I opened, meetings I attended (including notes), or research I do on new features.

My rule is that anything taking away my time should go into the worklog.

## What's the big deal?

I consider the worklog to be the most important document I keep regarding work.

- I use this worklog for every performance review and/or promotion document preparation since I can literally scan through it and pull out a list of the important stuff I did for a certain period amount of time.
- I cannot count the times I [grepped](https://en.wikipedia.org/wiki/Grep) the worklog for things I remember I did in the past, but vaguely remember exactly what it was or how it was solved. It's been invaluable!
- Occasionally, a colleague will ask about something I did and the answer will be in the worklog since information is spread in several internal tools making discovery hard.
- Those rainy days I mentioned above will come no matter how genius you are, and the worklog will be there for you.

## Want a tip?

Keep notes of everything you do throughout the day. **Embrace the worklog!**


In a previous article where I wrote about <a href="https://www.lambrospetrou.com/articles/digital-braindump-and-productivity-tools/">my favourite productivity tool</a> I briefly mentioned about my daily work log habbit. This article is its origin story.
<h2 id="origin">Origin</h2>
About 5 years ago I returned to Amazon (did an internship the year before) as a full-time employee after finishing my Masters degree. During one of my first 1:1 meetings with my manager we were talking about career growth and to this day I remember him saying the following.
<blockquote>
If I can only give you one tip, it’s to keep notes of what you do every day. Not for the usual reasons like performance reviews, but because there will be days that you will feel unproductive, thinking that you don’t produce anything, and it will make you feel bad. Those notes will be the only way to go back and see all the work you did during those days.
</blockquote>
This is by far the best tip I received in my (albeit short) career, and it’s the first tip I give to every new hire joining my team as well.
I call these notes the worklog, and it’s essentially a big text file with everything I do throughout the day in chronological order. Anything can go into the worklog like tasks or issues I worked on, code reviews I opened, meetings I attended (including notes), or research I do on new features.
My rule is that anything taking away my time should go into the worklog.
<h2 id="whats-the-big-deal">What’s the big deal?</h2>
I consider the worklog to be the most important document I keep regarding work.
<ul>
<li>I use this worklog for every performance review and/or promotion document preparation since I can literally scan through it and pull out a list of the important stuff I did for a certain period amount of time.</li>
<li>I cannot count the times I <a href="https://en.wikipedia.org/wiki/Grep">grepped</a> the worklog for things I remember I did in the past, but vaguely remember exactly what it was or how it was solved. It’s been invaluable!</li>
<li>Occasionally, a colleague will ask about something I did and the answer will be in the worklog since information is spread in several internal tools making discovery hard.</li>
<li>Those rainy days I mentioned above will come no matter how genius you are, and the worklog will be there for you.</li>
</ul>
<h2 id="want-a-tip">Want a tip?</h2>
Keep notes of everything you do throughout the day. Embrace the worklog!

The most important tip I would give to anyone is to have a daily work log containing every bit of work throughout a day.

Best tip I received — The worklog


I thought about writing this article many times over the years but I always thought it was not really something worthy of writing. However, after I saw the announcement for [Github Codespaces](https://github.com/features/codespaces) a few days ago I decided to write it because that announcement resonated so much with my daily workflow.

I will describe what I have been using for the past 5 years as my **productivity tool**, to keep my thoughts, notes, and ideas organised.

## What do I write down?

See below a non-exchaustive list of things that I take notes for over time:

- I write down everything I do during my day at work (**extremely crucial** and deserves an article on its own)
- Draft articles for my website
- Ideas for side projects
- Business ideas I think (and then reject)
- Material to read, watch or listen for several subjects
- Research I do on investments
- Notes for flats, houses, jobs when I am in the lookout
- Many many more things...

I am pretty sure everyone encounters at least a few of the items listed throughout the day.

There are many productivity tools available, especially mobile apps, that are supposed to simplify your life, but almost everything I used, and I tried hundreds of them, always has something that ruins it for me.

As a software engineer I spend a lot of time in text editors working with source code. At some point I realised that the best productivity tool was sitting right in front of me... That is **text files**.

In the old days people were using pen and paper to take notes, and many still do, including myself! Pen and paper is easy, quick, and everywhere.

In addition, one of the requirements of any productivity workflow is to have your notes available on all of your devices. The best ideas always come at the weirdest time and place.

## What do I use?

A text file is very similar with pen and paper. Text file editing is available on every device type and it's quick and easy to write anything.

File synchronization across devices is a solved problem nowadays with services like [Google Drive](https://drive.google.com/) and [Dropbox](http://dropbox.com/). In the software world we also have source code versioning software like [Git](https://git-scm.com/).

Many of the applications available restrict you in the type of notes you can take, the format to write them in, searchability might be limited, they have size limit restrictions, and they might not even support your device or operating system.

Having said all this... What do I use?

My productivity tool is basically a directory named `notes` consisting of several text files, organized sometimes in other sub-directories. Initially, I had this directory inside Dropbox, then Google Drive, and now in a [self-hosted git repository](https://www.lambrospetrou.com/articles/self-hosted-private-git/).

I love this approach because not only it's extremely easy to write down anything at any time, but I can also put additional files or material inside that directory without any restriction. All text editors provide amazing search capabilities, from simple file name searching, to fuzzy text search of the content of your notes.

You can write your notes in formats like [Markdown](https://guides.github.com/features/mastering-markdown/) or [AsciiDoc](https://asciidoctor.org/docs/what-is-asciidoc/) and have your editor render them nicely. Many editors also support displaying images or PDFs inline, and you have the full power of your device to open any type of file inside this directory. Again, there is no restriction of what files to put in your directory.

So, long story short:

- I use [Visual Studio Code](https://code.visualstudio.com/) as my editor rooted at the `notes` directory.
- I write most of my notes in Markdown (`.md`) format.
- I have one file named `scratchpad.md` which acts as my temporary canvas. This is probably my most edited file. I do anything temporary in here, from drafting medium-to-long emails before putting them in Outlook, writing long messages before pasting them in Slack/Mattermost/Facebook, and whatever else that needs a temporary space.
- I have my passwords inside [encrypted files](https://www.lambrospetrou.com/articles/encrypt-files-with-password-linux/).
- I use a self-hosted git repository, but I am considering moving to a [Github private repository](https://github.blog/2019-01-07-new-year-new-github/). Using git is powerful for me because I can immediately jump to any previous version of any file in the directory from the beginning of time.

### What else?

The approach described above covers almost the entirety of my needs. However, there are two things that I do in other tools.

1. Tiny notes that I need to take while on the move, i.e. walking to the train or the bus, or standing in the elevator, etc. I use [Google Keep](https://keep.google.com/) for this but any other lightweight note taking app with synchronization across devices works absolutely fine. Once I am on my desk, I move things into my `notes` directory as needed.
2. Calendar events. I still believe that events belong into a traditional calendar app providing core features like reminders and recurring events. I use [Google Calendar](http://calendar.google.com/).

## Conclusion

I know there are a million productivity tools, I tried most of them, and I also know that what I described is not rocket science distilled. It's not even something new, people have been using similar approaches for years. For example, I find [this Emacs Org-mode presentation](https://www.youtube.com/watch?v=oJTwQvgfgMM) from 12 years ago fascinating.

However, I do hope that this article will convince you to try a simple approach, or maybe combine 2-3 complimentary tools together to get the most out of your day. There is no need to force yourself to go all-in with an application that puts restrictions on you because it's supposed to be the best productivity tool of the month.

Now you can probably see why Github Codespaces triggerred my brain. My favourite editor will be available across all my devices (laptop, mobile, tablet) through a web browser allowing me to edit my notes directly (assuming I use Github) from anywhere 🚀


I thought about writing this article many times over the years but I always thought it was not really something worthy of writing. However, after I saw the announcement for <a href="https://github.com/features/codespaces">Github Codespaces</a> a few days ago I decided to write it because that announcement resonated so much with my daily workflow.
I will describe what I have been using for the past 5 years as my productivity tool, to keep my thoughts, notes, and ideas organised.
<h2 id="what-do-i-write-down">What do I write down?</h2>
See below a non-exchaustive list of things that I take notes for over time:
<ul>
<li>I write down everything I do during my day at work (extremely crucial and deserves an article on its own)</li>
<li>Draft articles for my website</li>
<li>Ideas for side projects</li>
<li>Business ideas I think (and then reject)</li>
<li>Material to read, watch or listen for several subjects</li>
<li>Research I do on investments</li>
<li>Notes for flats, houses, jobs when I am in the lookout</li>
<li>Many many more things…</li>
</ul>
I am pretty sure everyone encounters at least a few of the items listed throughout the day.
There are many productivity tools available, especially mobile apps, that are supposed to simplify your life, but almost everything I used, and I tried hundreds of them, always has something that ruins it for me.
As a software engineer I spend a lot of time in text editors working with source code. At some point I realised that the best productivity tool was sitting right in front of me… That is text files.
In the old days people were using pen and paper to take notes, and many still do, including myself! Pen and paper is easy, quick, and everywhere.
In addition, one of the requirements of any productivity workflow is to have your notes available on all of your devices. The best ideas always come at the weirdest time and place.
<h2 id="what-do-i-use">What do I use?</h2>
A text file is very similar with pen and paper. Text file editing is available on every device type and it’s quick and easy to write anything.
File synchronization across devices is a solved problem nowadays with services like <a href="https://drive.google.com/">Google Drive</a> and <a href="http://dropbox.com/">Dropbox</a>. In the software world we also have source code versioning software like <a href="https://git-scm.com/">Git</a>.
Many of the applications available restrict you in the type of notes you can take, the format to write them in, searchability might be limited, they have size limit restrictions, and they might not even support your device or operating system.
Having said all this… What do I use?
My productivity tool is basically a directory named <code>notes</code> consisting of several text files, organized sometimes in other sub-directories. Initially, I had this directory inside Dropbox, then Google Drive, and now in a <a href="https://www.lambrospetrou.com/articles/self-hosted-private-git/">self-hosted git repository</a>.
I love this approach because not only it’s extremely easy to write down anything at any time, but I can also put additional files or material inside that directory without any restriction. All text editors provide amazing search capabilities, from simple file name searching, to fuzzy text search of the content of your notes.
You can write your notes in formats like <a href="https://guides.github.com/features/mastering-markdown/">Markdown</a> or <a href="https://asciidoctor.org/docs/what-is-asciidoc/">AsciiDoc</a> and have your editor render them nicely. Many editors also support displaying images or PDFs inline, and you have the full power of your device to open any type of file inside this directory. Again, there is no restriction of what files to put in your directory.
So, long story short:
<ul>
<li>I use <a href="https://code.visualstudio.com/">Visual Studio Code</a> as my editor rooted at the <code>notes</code> directory.</li>
<li>I write most of my notes in Markdown (<code>.md</code>) format.</li>
<li>I have one file named <code>scratchpad.md</code> which acts as my temporary canvas. This is probably my most edited file. I do anything temporary in here, from drafting medium-to-long emails before putting them in Outlook, writing long messages before pasting them in Slack/Mattermost/Facebook, and whatever else that needs a temporary space.</li>
<li>I have my passwords inside <a href="https://www.lambrospetrou.com/articles/encrypt-files-with-password-linux/">encrypted files</a>.</li>
<li>I use a self-hosted git repository, but I am considering moving to a <a href="https://github.blog/2019-01-07-new-year-new-github/">Github private repository</a>. Using git is powerful for me because I can immediately jump to any previous version of any file in the directory from the beginning of time.</li>
</ul>
<h3 id="what-else">What else?</h3>
The approach described above covers almost the entirety of my needs. However, there are two things that I do in other tools.
<ol>
<li>Tiny notes that I need to take while on the move, i.e. walking to the train or the bus, or standing in the elevator, etc. I use <a href="https://keep.google.com/">Google Keep</a> for this but any other lightweight note taking app with synchronization across devices works absolutely fine. Once I am on my desk, I move things into my <code>notes</code> directory as needed.</li>
<li>Calendar events. I still believe that events belong into a traditional calendar app providing core features like reminders and recurring events. I use <a href="http://calendar.google.com/">Google Calendar</a>.</li>
</ol>
<h2 id="conclusion">Conclusion</h2>
I know there are a million productivity tools, I tried most of them, and I also know that what I described is not rocket science distilled. It’s not even something new, people have been using similar approaches for years. For example, I find <a href="https://www.youtube.com/watch?v=oJTwQvgfgMM">this Emacs Org-mode presentation</a> from 12 years ago fascinating.
However, I do hope that this article will convince you to try a simple approach, or maybe combine 2-3 complimentary tools together to get the most out of your day. There is no need to force yourself to go all-in with an application that puts restrictions on you because it’s supposed to be the best productivity tool of the month.
Now you can probably see why Github Codespaces triggerred my brain. My favourite editor will be available across all my devices (laptop, mobile, tablet) through a web browser allowing me to edit my notes directly (assuming I use Github) from anywhere 🚀

The best productivity tool I use to hold my digital braindump is plain and simple text files synchronized across devices.

My digital braindump and productivity tools

## Overview

If you [follow me](https://twitter.com/LambrosPetrou) online you probably know that I like serverless platforms a lot. Serverless nowadays is a pretty ambiguous term and can mean anything; from [SQLite coining the term](https://www.sqlite.org/serverless.html) to [AWS enumerating their serverless offerings](https://aws.amazon.com/serverless/).

In this article, serverless corresponds to a single page application (SPA) website that is accompanied by a lightweight API. This setup exploded in popularity recently, and it even got a name, [Jamstack](https://jamstack.org/), for **J**avaScript, **A**PIs, and **M**arkup.

I experimented with many platforms over the past few years and I am going to briefly go over my current top 3 platforms to deploy your application, Netlify, Vercel, and AWS.

### TL;DR

All three platforms are great and I recommend all of them, depending on how much time you are willing to spend, and how much extensibility you think you might need in the future.

## Netlify

Website: <https://netlify.com>

I discovered Netlify couple of years ago, and have been tracking their progress ever since. The first time I saw it, I was simply stunned by how simple and fun it was to use while at the same time having powerful features.

Netlify has a lot of features and it feels more like an ecosystem of pluggable functionality (what they call add-ons), with flexible pricing.

Most notable add-on features:
- Serverless Functions
- Instant-forms
- Identity
- Analytics

The core product of the Netlify Platform is the combination of **Netlify Build** and **Netlify Edge**. Netlify Build is the ability to easily connect your Netlify project to your code repository (Github, Gitlab, BitBucket) and deploy your changes after every commit with a unique URL for each deployment. Netlify Edge is the application delivery network (ADN) which propagates the project's artifacts in locations across the globe, similar to a normal content delivery network (CDN) but much smarter, and faster.

The whole process to get started is so simple that you can [just drag-and-drop](https://app.netlify.com/drop) your project folder onto their website and deploy it in seconds!

I cannot possibly enumerate all the features Netlify provides but for the sake of this article we will focus on the core platform and the [Serverless Functions add-on](https://docs.netlify.com/functions/overview/).

Serverless functions use [AWS Lambda](https://aws.amazon.com/lambda/) behind the scenes, but abstract it away so that we don't have to fiddle with API Gateway, IAM role permissions, and all the nitty gritty AWS boilerplate.

For example, if we take JavaScript as an example, simply creating a file `functions/hello-world.js` with the content below will create an API accessible at `/.netlify/functions/hello-world`.

```javascript
exports.handler = function(event, context, callback) {
 callback(null, {
 statusCode: 200,
 body: "Hello, World"
 });
}
```

That's it for Netlify, let's move on to the next one.

## Vercel

Website: <https://vercel.com>

Vercel was [until recently known as Zeit Now](https://vercel.com/blog/zeit-is-now-vercel), and is extremely similar to Netlify in terms of target audience. Vercel however puts a lot of emphasis on their **zero config** deployments, and you can see it mentioned all over their website and docs. By zero-config deployment we mean that their system is [trying to be smart and guess the build system or framework that your project is using](https://vercel.com/docs/v2/build-step) based on your files and automatically do what it needs to do without you specifying anything. It works very well most of the time, apart from a [small issue](https://github.com/zeit/now/discussions/4132) I discovered with the custom build system.

Vercel provides a similar experience as Netlify, where you can connect your repository and instantly build and deploy your project after every commit, and also includes a delivery network.

A big feature is once again their [Serverless Functions](https://vercel.com/docs/v2/serverless-functions/introduction) offering, which is also using AWS Lambda under the covers. However, Vercel is a step up from Netlify, with [more languages](https://vercel.com/docs/v2/serverless-functions/supported-languages) and [more regions supported](https://vercel.com/docs/v2/edge-network/regions).

Its delivery network is also quite powerful, and more feature-rich than Netlify's, since apart from the static assets, it can also [cache serverless function responses](https://vercel.com/docs/v2/serverless-functions/edge-caching).

Just for completeness, a serverless function example in JavaScript requires the following content in the file `api/hello.js` in order to expose the API at `/api/hello?name=xxx`.

```javascript
module.exports = (req, res) => {
 const { name = 'World' } = req.query
 res.status(200).send(`Hello ${name}!`)
}
```

As you can see, even though it's based on AWS Lambda, Vercel decided to use custom function signatures for the handler in contrary to Netlify which is using AWS's format.

Before we finish with Vercel, I would like to briefly mention [Next.js](https://nextjs.org/), the React framework they developed which is **simply amazing**. I recently migrated my blog to use this and I cannot emphasize [how great it is](https://nextjs.org/blog/next-9-3#next-gen-static-site-generation-ssg-support), and in conjunction with Vercel's platform they make a killer combination 🚀

## Amazon Web Services (AWS)

Even though AWS does not provide a nice coherent Jamstack platform (I don't like [AWS Amplify](https://aws.amazon.com/amplify/) at all), it provides all the necessary services to build your application.

For the past 5 years I have been using this myself, and the main services we need are [Amazon S3](https://aws.amazon.com/s3/) for storage of the static assets, [Amazon CloudFront](https://aws.amazon.com/cloudfront/) as our CDN, and [AWS Lambda](https://aws.amazon.com/lambda/) with [API Gateway](https://aws.amazon.com/api-gateway/) for our serverless functions API.

AWS has an advantage over both Netlify and Vercel, because of [Lambda@Edge](https://aws.amazon.com/lambda/edge/) which is basically a slightly restricted version of AWS Lambda running on the edge locations of Amazon CloudFront, so much closer to the customers compared to the normal Lambda functions which are in the regional datacenters. I have been using Lambda@Edge for years now, both in personal projects but also while I was working at Amazon and I love it!

As you can see, it involves more moving pieces but they are all super robust services used by thousands of customers, serving billions of requests every year without issues. Some of the AWS services themselves are being built on-top of these services, which proves their reliability and that AWS bets on them working as expected!

Finally, I would strongly recommend using [AWS CDK](https://aws.amazon.com/cdk/) to provision your resources for all the above services in code, referred to as Infrastructure as Code (remember CloudFormation?).

## Conclusion

I only scratched the surface of each platform's feature set, however it's quite clear that some are better at something and worse at something else.

If you focus only on Jamstack applications, my recommendation would be to go with Vercel. It has amazing performance, and the zero-config approach really does wonders for the majority of the popular frameworks. Recommending [Next.js](https://nextjs.org/) for one more time 😃

If you are going to find uses for Netlify's add-ons, then it's a great choice as well! Really, Netlify and Vercel are very similar and you cannot go wrong with either. They even published an article yesterday on [how to deploy a Next.js application on Netlify](https://www.netlify.com/blog/2020/05/04/building-a-markdown-blog-with-next-9.3-and-netlify/).

Finally, if your application is going to need additional cloud infrastructure to support it, like queueing systems, databases, detailed monitoring, or anything else not provided by these Jamstack oriented platforms, then AWS is probably your best bet. You can always make requests from Netlify/Vercel's serverless functions to other AWS resources, but it's a matter of control.

That's it for this article, maybe I will do a deep-dive in the future for certain features...

<h2 id="overview">Overview</h2>
If you <a href="https://twitter.com/LambrosPetrou">follow me</a> online you probably know that I like serverless platforms a lot. Serverless nowadays is a pretty ambiguous term and can mean anything; from <a href="https://www.sqlite.org/serverless.html">SQLite coining the term</a> to <a href="https://aws.amazon.com/serverless/">AWS enumerating their serverless offerings</a>.
In this article, serverless corresponds to a single page application (SPA) website that is accompanied by a lightweight API. This setup exploded in popularity recently, and it even got a name, <a href="https://jamstack.org/">Jamstack</a>, for JavaScript, APIs, and Markup.
I experimented with many platforms over the past few years and I am going to briefly go over my current top 3 platforms to deploy your application, Netlify, Vercel, and AWS.
<h3 id="tldr">TL;DR</h3>
All three platforms are great and I recommend all of them, depending on how much time you are willing to spend, and how much extensibility you think you might need in the future.
<h2 id="netlify">Netlify</h2>
Website: <a href="https://netlify.com">https://netlify.com</a>
I discovered Netlify couple of years ago, and have been tracking their progress ever since. The first time I saw it, I was simply stunned by how simple and fun it was to use while at the same time having powerful features.
Netlify has a lot of features and it feels more like an ecosystem of pluggable functionality (what they call add-ons), with flexible pricing.
Most notable add-on features:
<ul>
<li>Serverless Functions</li>
<li>Instant-forms</li>
<li>Identity</li>
<li>Analytics</li>
</ul>
The core product of the Netlify Platform is the combination of Netlify Build and Netlify Edge. Netlify Build is the ability to easily connect your Netlify project to your code repository (Github, Gitlab, BitBucket) and deploy your changes after every commit with a unique URL for each deployment. Netlify Edge is the application delivery network (ADN) which propagates the project’s artifacts in locations across the globe, similar to a normal content delivery network (CDN) but much smarter, and faster.
The whole process to get started is so simple that you can <a href="https://app.netlify.com/drop">just drag-and-drop</a> your project folder onto their website and deploy it in seconds!
I cannot possibly enumerate all the features Netlify provides but for the sake of this article we will focus on the core platform and the <a href="https://docs.netlify.com/functions/overview/">Serverless Functions add-on</a>.
Serverless functions use <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> behind the scenes, but abstract it away so that we don’t have to fiddle with API Gateway, IAM role permissions, and all the nitty gritty AWS boilerplate.
For example, if we take JavaScript as an example, simply creating a file <code>functions/hello-world.js</code> with the content below will create an API accessible at <code>/.netlify/functions/hello-world</code>.
<pre><code class="language-javascript">exports.handler = function(event, context, callback) {
 callback(null, {
 statusCode: 200,
 body: &quot;Hello, World&quot;
 });
}</code></pre>
That’s it for Netlify, let’s move on to the next one.
<h2 id="vercel">Vercel</h2>
Website: <a href="https://vercel.com">https://vercel.com</a>
Vercel was <a href="https://vercel.com/blog/zeit-is-now-vercel">until recently known as Zeit Now</a>, and is extremely similar to Netlify in terms of target audience. Vercel however puts a lot of emphasis on their zero config deployments, and you can see it mentioned all over their website and docs. By zero-config deployment we mean that their system is <a href="https://vercel.com/docs/v2/build-step">trying to be smart and guess the build system or framework that your project is using</a> based on your files and automatically do what it needs to do without you specifying anything. It works very well most of the time, apart from a <a href="https://github.com/zeit/now/discussions/4132">small issue</a> I discovered with the custom build system.
Vercel provides a similar experience as Netlify, where you can connect your repository and instantly build and deploy your project after every commit, and also includes a delivery network.
A big feature is once again their <a href="https://vercel.com/docs/v2/serverless-functions/introduction">Serverless Functions</a> offering, which is also using AWS Lambda under the covers. However, Vercel is a step up from Netlify, with <a href="https://vercel.com/docs/v2/serverless-functions/supported-languages">more languages</a> and <a href="https://vercel.com/docs/v2/edge-network/regions">more regions supported</a>.
Its delivery network is also quite powerful, and more feature-rich than Netlify’s, since apart from the static assets, it can also <a href="https://vercel.com/docs/v2/serverless-functions/edge-caching">cache serverless function responses</a>.
Just for completeness, a serverless function example in JavaScript requires the following content in the file <code>api/hello.js</code> in order to expose the API at <code>/api/hello?name=xxx</code>.
<pre><code class="language-javascript">module.exports = (req, res) =&gt; {
 const { name = &#39;World&#39; } = req.query
 res.status(200).send(`Hello ${name}!`)
}</code></pre>
As you can see, even though it’s based on AWS Lambda, Vercel decided to use custom function signatures for the handler in contrary to Netlify which is using AWS’s format.
Before we finish with Vercel, I would like to briefly mention <a href="https://nextjs.org/">Next.js</a>, the React framework they developed which is simply amazing. I recently migrated my blog to use this and I cannot emphasize <a href="https://nextjs.org/blog/next-9-3#next-gen-static-site-generation-ssg-support">how great it is</a>, and in conjunction with Vercel’s platform they make a killer combination 🚀
<h2 id="amazon-web-services-aws">Amazon Web Services (AWS)</h2>
Even though AWS does not provide a nice coherent Jamstack platform (I don’t like <a href="https://aws.amazon.com/amplify/">AWS Amplify</a> at all), it provides all the necessary services to build your application.
For the past 5 years I have been using this myself, and the main services we need are <a href="https://aws.amazon.com/s3/">Amazon S3</a> for storage of the static assets, <a href="https://aws.amazon.com/cloudfront/">Amazon CloudFront</a> as our CDN, and <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> with <a href="https://aws.amazon.com/api-gateway/">API Gateway</a> for our serverless functions API.
AWS has an advantage over both Netlify and Vercel, because of <a href="https://aws.amazon.com/lambda/edge/">Lambda@Edge</a> which is basically a slightly restricted version of AWS Lambda running on the edge locations of Amazon CloudFront, so much closer to the customers compared to the normal Lambda functions which are in the regional datacenters. I have been using Lambda@Edge for years now, both in personal projects but also while I was working at Amazon and I love it!
As you can see, it involves more moving pieces but they are all super robust services used by thousands of customers, serving billions of requests every year without issues. Some of the AWS services themselves are being built on-top of these services, which proves their reliability and that AWS bets on them working as expected!
Finally, I would strongly recommend using <a href="https://aws.amazon.com/cdk/">AWS CDK</a> to provision your resources for all the above services in code, referred to as Infrastructure as Code (remember CloudFormation?).
<h2 id="conclusion">Conclusion</h2>
I only scratched the surface of each platform’s feature set, however it’s quite clear that some are better at something and worse at something else.
If you focus only on Jamstack applications, my recommendation would be to go with Vercel. It has amazing performance, and the zero-config approach really does wonders for the majority of the popular frameworks. Recommending <a href="https://nextjs.org/">Next.js</a> for one more time 😃
If you are going to find uses for Netlify’s add-ons, then it’s a great choice as well! Really, Netlify and Vercel are very similar and you cannot go wrong with either. They even published an article yesterday on <a href="https://www.netlify.com/blog/2020/05/04/building-a-markdown-blog-with-next-9.3-and-netlify/">how to deploy a Next.js application on Netlify</a>.
Finally, if your application is going to need additional cloud infrastructure to support it, like queueing systems, databases, detailed monitoring, or anything else not provided by these Jamstack oriented platforms, then AWS is probably your best bet. You can always make requests from Netlify/Vercel’s serverless functions to other AWS resources, but it’s a matter of control.
That’s it for this article, maybe I will do a deep-dive in the future for certain features…

A quick overview of three Jamstack platforms, Netlify, Vercel (formerly Zeit Now), and AWS.

Battle of the Jamstack platforms — Netlify, Vercel, AWS


## Overview

Over the past few years I played around with many languages, starting with C/C++ at college to Java and JavaScript professionally, and to many others during my own time out of interest like Go, Scala, Python, Elixir, Clojure, Racket, Ruby and more.

During the last year I wanted to focus more on functional languages, specifically [Elixir](http://elixir-lang.org/) and [Clojure](https://clojure.org/). Unfortunately, I didn't manage to do much due to work and other things taking up my time, but I really loved both languages.

After trying and playing with many languages I realised that there are certain things I am looking for in order to keep me interested for more than a few days.

My hobby language needs to:
- Be fun to write
- Have a good ecosystem to manage projects without a lot of ceremony
- Be general purpose enough to write scripts and CLIs, but also web servers/clients
- Be fast enough
- Optional - Functional programming
- Optional - Concurrency and Parallelism constructs

Based on the above requirements, this year I decided to focus on the following languages, and hopefully I will be able to spend a few months with each one and do some projects with them.

## ReasonML

I had [OCaml](https://ocaml.org/) on my to-dive-deep list for years, but never jumped into it.
When Facebook released [ReasonML](https://reasonml.github.io/) a while ago I thought it was a nice bridge/opportunity to get me into the OCaml world.

It's no secret that I love the [Node](https://nodejs.org/) ecosystem, although it's something most people hate these days. I truly believe that when you learn how to navigate around without getting disoriented by the huge amount of available modules, it provides a lot of things that other languages lack in a simple and easy way to use!

ReasonML [has a fantastic integration with NPM](https://reasonml.github.io/docs/en/installation) which makes things easy.

The first project I am working on with ReasonML is to write an interpreter for the [Monkey language](https://monkeylang.org/), following the book [Writing an Interpreter in Go](https://interpreterbook.com/) by Thorsten Ball. I highly recommend this book, it's simply awesome!


## ClojureScript

Around 2 years ago I played around with Lisps, specifically [Racket](https://racket-lang.org/) and Clojure. 

Racket was surprisingly refreshing and I really enjoyed learning it, with great libraries and amazing documentation. However, it was quite slow for some things I did back then, but I know that some work was done to make it faster nowadays so I need to revisit it at some point.

Clojure, as a language trully impressed me. It has everything a functional language needs, with core and efficient datastructures, and ideas that are very powerful once learnt! However, although I have been using JVM languages professionally at work for 5+ years, I find the ecosystem to require too much ceremony to do the simplest thing, so I avoid it for my hobby projects.

[ClojureScript](https://clojurescript.org/) on the other hand, is the same (almost) language as Clojure, but compiles to JavaScript. This means that you can use the NPM ecosystem to consume and produce software written in ClojureScript alongside JavaScript.

In addition, since [Clojure has a very slow startup](http://clojure-goes-fast.com/blog/clojures-slow-start/) due to the way it handles class loading it's not really suitable for quick scripts I write every now and then, whereas ClojureScript can use Node as its runtime leading to zero startup latency.

There are some amazing projects to make the ClojureScript integration with NPM easier, like [Shadow CLJS](https://shadow-cljs.github.io/docs/UsersGuide.html) which is my go-to; I even wrote a project template generator for it, [create-shadow-cljs-app](https://github.com/lambrospetrou/create-shadow-cljs-app). Coincidentally, David Nollen also wrote an article yesterday about a new [feature in ClojureScript to integrate better with JavaScript bundlers](https://clojurescript.org/guides/webpack) which will hopefully bring more people.

Clojure(Script) is so nice that the community wrote standalone REPLs to make it usable directly from the command line for scripting. My favourite is [planck-repl](https://github.com/planck-repl/planck), but [babashka](https://github.com/borkdude/babashka) has been making the rounds recently as well and I will have to look into it.

I wrote a few small things in ClojureScript but I want to write something bigger to get a better feel of the language itself.

I will definitly have hard time deciding between ReasonML and ClojureScript since they both target Node, both can be used for similar things, and they are both functional languages. However, while they are very similar in some aspects, they are two very different languages, coming from different language families, Lisp vs ML.

## Go

I started writing in [Go](https://golang.org/) back at 2014, and since then I wrote scripts, API servers, several static site generators for my website and other tools.

The language itself has nothing extraordinary, but it really feels nice using it and you can get many things done just by using the standard library, which is amazing on its own.

In addition, being a compiled language means it runs very fast, and it has excellent concurrency constructs that make writing concurrent software very easy, which is nice for the situations where you need to put those CPU cores into work.

I tried replacing Go with some other language over the past few years to be my quick go-to language but I always come back. Maybe ReasonML or ClojureScript will do the trick, but the simplicity, the performance, and the concurrency features are really unmatched so far.

Even though I already wrote several things in Go, I want to get back to it this year since I haven't used it a lot recently.

## Others

As I said in the overview, I really like playing around with many languages. Some of them lose me from the hello world, but many of them end up in my to-dive-deep list.

Other languages I really liked and I would like to spend more time with them, probably not a lot this year, are Elixir and [Rust](https://www.rust-lang.org/).

Elixir has everything I love in a language apart from single-threaded performance really. It's amazing for anything that requires a lot of juggling around since it supports millions of processes running independently, with their own garbage collection, preemptive scheduling, and other great features provided by its runtime, the [BEAM VM](https://blog.stenmans.org/theBeamBook/). As a language, and as an ecosystem it is very high on my list! There is some amazing work being done now [with Phoenix LiveViews](https://www.phoenixframework.org/blog/build-a-real-time-twitter-clone-in-15-minutes-with-live-view-and-phoenix-1-5) which is very interesting, and I am eager to see where it will lead.

Rust is making waves recently a lot! Both in my social circles, but also professionally it gets adopted by many big companies for their performance-critical systems. It has a nice combination of functional programming concepts together with performance oriented features like memory management to avoid runtime issues. In addition, it is among the best languages to use in order to compile to [WebAssembly](https://www.rust-lang.org/what/wasm) which is something I am extremely interested in.

I wish the days were longer :) Much to learn, so little time...


<h2 id="overview">Overview</h2>
Over the past few years I played around with many languages, starting with C/C++ at college to Java and JavaScript professionally, and to many others during my own time out of interest like Go, Scala, Python, Elixir, Clojure, Racket, Ruby and more.
During the last year I wanted to focus more on functional languages, specifically <a href="http://elixir-lang.org/">Elixir</a> and <a href="https://clojure.org/">Clojure</a>. Unfortunately, I didn’t manage to do much due to work and other things taking up my time, but I really loved both languages.
After trying and playing with many languages I realised that there are certain things I am looking for in order to keep me interested for more than a few days.
My hobby language needs to:
<ul>
<li>Be fun to write</li>
<li>Have a good ecosystem to manage projects without a lot of ceremony</li>
<li>Be general purpose enough to write scripts and CLIs, but also web servers/clients</li>
<li>Be fast enough</li>
<li>Optional - Functional programming</li>
<li>Optional - Concurrency and Parallelism constructs</li>
</ul>
Based on the above requirements, this year I decided to focus on the following languages, and hopefully I will be able to spend a few months with each one and do some projects with them.
<h2 id="reasonml">ReasonML</h2>
I had <a href="https://ocaml.org/">OCaml</a> on my to-dive-deep list for years, but never jumped into it.
When Facebook released <a href="https://reasonml.github.io/">ReasonML</a> a while ago I thought it was a nice bridge/opportunity to get me into the OCaml world.
It’s no secret that I love the <a href="https://nodejs.org/">Node</a> ecosystem, although it’s something most people hate these days. I truly believe that when you learn how to navigate around without getting disoriented by the huge amount of available modules, it provides a lot of things that other languages lack in a simple and easy way to use!
ReasonML <a href="https://reasonml.github.io/docs/en/installation">has a fantastic integration with NPM</a> which makes things easy.
The first project I am working on with ReasonML is to write an interpreter for the <a href="https://monkeylang.org/">Monkey language</a>, following the book <a href="https://interpreterbook.com/">Writing an Interpreter in Go</a> by Thorsten Ball. I highly recommend this book, it’s simply awesome!
<h2 id="clojurescript">ClojureScript</h2>
Around 2 years ago I played around with Lisps, specifically <a href="https://racket-lang.org/">Racket</a> and Clojure. 
Racket was surprisingly refreshing and I really enjoyed learning it, with great libraries and amazing documentation. However, it was quite slow for some things I did back then, but I know that some work was done to make it faster nowadays so I need to revisit it at some point.
Clojure, as a language trully impressed me. It has everything a functional language needs, with core and efficient datastructures, and ideas that are very powerful once learnt! However, although I have been using JVM languages professionally at work for 5+ years, I find the ecosystem to require too much ceremony to do the simplest thing, so I avoid it for my hobby projects.
<a href="https://clojurescript.org/">ClojureScript</a> on the other hand, is the same (almost) language as Clojure, but compiles to JavaScript. This means that you can use the NPM ecosystem to consume and produce software written in ClojureScript alongside JavaScript.
In addition, since <a href="http://clojure-goes-fast.com/blog/clojures-slow-start/">Clojure has a very slow startup</a> due to the way it handles class loading it’s not really suitable for quick scripts I write every now and then, whereas ClojureScript can use Node as its runtime leading to zero startup latency.
There are some amazing projects to make the ClojureScript integration with NPM easier, like <a href="https://shadow-cljs.github.io/docs/UsersGuide.html">Shadow CLJS</a> which is my go-to; I even wrote a project template generator for it, <a href="https://github.com/lambrospetrou/create-shadow-cljs-app">create-shadow-cljs-app</a>. Coincidentally, David Nollen also wrote an article yesterday about a new <a href="https://clojurescript.org/guides/webpack">feature in ClojureScript to integrate better with JavaScript bundlers</a> which will hopefully bring more people.
Clojure(Script) is so nice that the community wrote standalone REPLs to make it usable directly from the command line for scripting. My favourite is <a href="https://github.com/planck-repl/planck">planck-repl</a>, but <a href="https://github.com/borkdude/babashka">babashka</a> has been making the rounds recently as well and I will have to look into it.
I wrote a few small things in ClojureScript but I want to write something bigger to get a better feel of the language itself.
I will definitly have hard time deciding between ReasonML and ClojureScript since they both target Node, both can be used for similar things, and they are both functional languages. However, while they are very similar in some aspects, they are two very different languages, coming from different language families, Lisp vs ML.
<h2 id="go">Go</h2>
I started writing in <a href="https://golang.org/">Go</a> back at 2014, and since then I wrote scripts, API servers, several static site generators for my website and other tools.
The language itself has nothing extraordinary, but it really feels nice using it and you can get many things done just by using the standard library, which is amazing on its own.
In addition, being a compiled language means it runs very fast, and it has excellent concurrency constructs that make writing concurrent software very easy, which is nice for the situations where you need to put those CPU cores into work.
I tried replacing Go with some other language over the past few years to be my quick go-to language but I always come back. Maybe ReasonML or ClojureScript will do the trick, but the simplicity, the performance, and the concurrency features are really unmatched so far.
Even though I already wrote several things in Go, I want to get back to it this year since I haven’t used it a lot recently.
<h2 id="others">Others</h2>
As I said in the overview, I really like playing around with many languages. Some of them lose me from the hello world, but many of them end up in my to-dive-deep list.
Other languages I really liked and I would like to spend more time with them, probably not a lot this year, are Elixir and <a href="https://www.rust-lang.org/">Rust</a>.
Elixir has everything I love in a language apart from single-threaded performance really. It’s amazing for anything that requires a lot of juggling around since it supports millions of processes running independently, with their own garbage collection, preemptive scheduling, and other great features provided by its runtime, the <a href="https://blog.stenmans.org/theBeamBook/">BEAM VM</a>. As a language, and as an ecosystem it is very high on my list! There is some amazing work being done now <a href="https://www.phoenixframework.org/blog/build-a-real-time-twitter-clone-in-15-minutes-with-live-view-and-phoenix-1-5">with Phoenix LiveViews</a> which is very interesting, and I am eager to see where it will lead.
Rust is making waves recently a lot! Both in my social circles, but also professionally it gets adopted by many big companies for their performance-critical systems. It has a nice combination of functional programming concepts together with performance oriented features like memory management to avoid runtime issues. In addition, it is among the best languages to use in order to compile to <a href="https://www.rust-lang.org/what/wasm">WebAssembly</a> which is something I am extremely interested in.
I wish the days were longer :) Much to learn, so little time…

The languages I will use for my hobby projects in 2020. ReasonML, ClojureScript and Go.

Hobby Languages for 2020


## Overview

After almost 5 years at Amazon, I really think that the [Amazon Leadership Principles](https://www.amazon.jobs/en-gb/principles) are among the few things that are always going to be engraved in me.

In the yearly reviews (so-called Forte) we choose up to 3 LPs for some of our colleagues to describe their superpowers and their growth opportunities, so in a similar fashion I will enumerate the 3 LPs that I consider to be the most crucial, and which have been the barometer to everything I was doing.

## Customer Obsession

>Leaders start with the customer and work backwards. They work vigorously to earn and keep customer trust. Although leaders pay attention to competitors, they obsess over customers.

Not much to say... Customers come first, so the customer should be the sole focus of anything you do. Choosing the right problems to solve, making the right decision between trade-offs, and delivering the final solution should all start and finish with the customer in mind.

## Insist on the Highest Standards

>Leaders have relentlessly high standards — many people may think these standards are unreasonably high. Leaders are continually raising the bar and driving their teams to deliver high quality products, services, and processes. Leaders ensure that defects do not get sent down the line and that problems are fixed so they stay fixed.

I cannot emphasize this enough. **Keeping the bar high**, and always insisting on high standards is not only important for the short-term but it's absolutely crucial for the long-term.

We face the temptation of doing something quick-and-dirty pretty much in every line of code we write or decision we make. Choosing this path enough times though means that the product will degrade over time, but even worse, we make the job of our future self or future colleagues super hard.

How many times did you want to _have two words_ with the author of some part of a system you worked on... :)

## Deliver Results

>Leaders focus on the key inputs for their business and deliver them with the right quality and in a timely fashion. Despite setbacks, they rise to the occasion and never compromise.

This is the culmination of all the Leadership Principles together in my opinion. You have to focus on the **key inputs for your business**, and deliver them with the **right quality**, and in a **timely fashion**, while at the same time **never compromising**.

Yeah, I pretty much re-stated the same thing, but this is the gist of it. At the end of the day, you have to deliver something to your customers, but the difficulty lies in doing it in a timely fashion without compromosing quality.

## Conclusion

**I honestly think that all 14 (as of now) LPs are super important!**

Having said that, it's almost impossible for every individual to have the same definition and the same bar for all 14 of them, and this is something I have seen a lot over the past few years. Each person will interpret each LP in their own way, which is fair and expected. 

The magic though happens when a team does not lose focus on the Leadership Principles and always comes back and re-evaluates everything against them!

I am glad I had the chance to be an Amazonian!



<h2 id="overview">Overview</h2>
After almost 5 years at Amazon, I really think that the <a href="https://www.amazon.jobs/en-gb/principles">Amazon Leadership Principles</a> are among the few things that are always going to be engraved in me.
In the yearly reviews (so-called Forte) we choose up to 3 LPs for some of our colleagues to describe their superpowers and their growth opportunities, so in a similar fashion I will enumerate the 3 LPs that I consider to be the most crucial, and which have been the barometer to everything I was doing.
<h2 id="customer-obsession">Customer Obsession</h2>
<blockquote>
Leaders start with the customer and work backwards. They work vigorously to earn and keep customer trust. Although leaders pay attention to competitors, they obsess over customers.
</blockquote>
Not much to say… Customers come first, so the customer should be the sole focus of anything you do. Choosing the right problems to solve, making the right decision between trade-offs, and delivering the final solution should all start and finish with the customer in mind.
<h2 id="insist-on-the-highest-standards">Insist on the Highest Standards</h2>
<blockquote>
Leaders have relentlessly high standards — many people may think these standards are unreasonably high. Leaders are continually raising the bar and driving their teams to deliver high quality products, services, and processes. Leaders ensure that defects do not get sent down the line and that problems are fixed so they stay fixed.
</blockquote>
I cannot emphasize this enough. Keeping the bar high, and always insisting on high standards is not only important for the short-term but it’s absolutely crucial for the long-term.
We face the temptation of doing something quick-and-dirty pretty much in every line of code we write or decision we make. Choosing this path enough times though means that the product will degrade over time, but even worse, we make the job of our future self or future colleagues super hard.
How many times did you want to have two words with the author of some part of a system you worked on… :)
<h2 id="deliver-results">Deliver Results</h2>
<blockquote>
Leaders focus on the key inputs for their business and deliver them with the right quality and in a timely fashion. Despite setbacks, they rise to the occasion and never compromise.
</blockquote>
This is the culmination of all the Leadership Principles together in my opinion. You have to focus on the key inputs for your business, and deliver them with the right quality, and in a timely fashion, while at the same time never compromising.
Yeah, I pretty much re-stated the same thing, but this is the gist of it. At the end of the day, you have to deliver something to your customers, but the difficulty lies in doing it in a timely fashion without compromosing quality.
<h2 id="conclusion">Conclusion</h2>
I honestly think that all 14 (as of now) LPs are super important!
Having said that, it’s almost impossible for every individual to have the same definition and the same bar for all 14 of them, and this is something I have seen a lot over the past few years. Each person will interpret each LP in their own way, which is fair and expected. 
The magic though happens when a team does not lose focus on the Leadership Principles and always comes back and re-evaluates everything against them!
I am glad I had the chance to be an Amazonian!

Amazon Leadership Principles — Choose 3

## Problem

After using [ReactJS](https://reactjs.org/) for two years, along with all its ecosystem ([Redux](https://redux.js.org/) for state management, [Redux-saga](https://redux-saga.js.org/) for managing side-effects, etc.) I wanted to experiment with a simpler approach where I was not forced to use many framework specific libraries, but at the same time having the same uni-directional flow of data and the ease of UI creation using components.

A few months ago, I came across [Mithril](https://mithril.js.org/), a lightweight component based framework which uses pure Javascript objects for its components. I was instantly hooked and wanted to use it in all my personal projects.

_**How can I get the state management I want without being tied to a framework so that I can use Mithril or React interchangeably without issues?**_

**TL;DR**

My answer is using [Meiosis pattern](https://meiosis.js.org/).

## Streams

In order to properly understand the solution let's go over some basics regarding streams.

There are several Javascript stream libraries but for the rest of the article I will use [Mithril/stream](https://mithril.js.org/stream.html).

Meiosis requires three basic operations on streams:

1. [Updating the stream value](https://mithril.js.org/stream.html#streams-as-variables)
2. [Mapping over the stream](https://mithril.js.org/stream.html#streammap) in order to listen for updates to the stream's value
3. [Scanning over the stream](https://mithril.js.org/stream.html#streamscan) in order to calculate accumulated value after every update

### Updating the stream value

```javascript
import * as ms from 'mithril/stream'

const s = ms()
s(1)
console.log(s()) // 1
s(s()+10)
console.log(s()) // 11
```

The example above shows that just calling a stream as a function will return the latest value of the stream, and supplying a value to the function application will update the stream's value.

Also, the fact that the stream itself is a function can be super useful since it can be composed into more complex functions.

### Stream.map

```javascript
import * as ms from 'mithril/stream'

const s = ms()

s.map(value => console.log(`updated value: ${value}`))

s(1) // prints "updated value: 1"
s(20) // prints "updated value: 20"
s(311) // prints "updated value: 311"
```

The [Stream.map](https://mithril.js.org/stream.html#streammap) operation is simple. It just executes a given function every time the stream value is updated, and also returns a new stream. The return value of the given function after every execution becomes the new value for the stream returned by the call to `map()`.

### Stream.scan

```javascript
import * as ms from 'mithril/stream'

const accumulate = (accumulator, value) => accumulator * value
const processValue = value => console.log(value)

const s = ms()
const scanned = ms.scan(accumulate, 1, s)
scanned.map(processValue)

s(1) // prints 1
s(2) // prints 2
s(5) // prints 10
s(100) // prints 1000
s(0.5) // prints 500
```

The distilled functionality of [Stream.scan](https://mithril.js.org/stream.html#streamscan) is that whenever the value of the source stream (`s`) changes, the `accumulate` function is called being passed the current accumulator value and the new source stream value. The return value will become the new value of the `scanned` stream and the accumulator value being passed to the next `accumulate` invocation.

The fascinating idea here is that a stream can have functions as its values.

In the following example we pass functions as values and as a result we can apply different operations on the accumulator value each time.

```javascript
import * as ms from 'mithril/stream'

const update = (accumulator, fn) => fn(accumulator)
const processValue = value => console.log(value)

const s = ms()
const scanned = ms.scan(update, 1, s)
scanned.map(processValue)

s(value => value + 1) // prints 2
s(value => value - 1) // prints 1
s(value => value * 10) // prints 10
s(value => 1234) // prints 1234
s(value => value / 2) // prints 617
```

Spend a few minutes to **fully** understand the above example because it's the bread and butter of the Meiosis pattern.

Notice how we pass functions in the stream and by the use of `scan` they are applied in order transforming the stream accumulated value.

## Meiosis Pattern

The [Meiosis website](https://meiosis.js.org/) is doing an amazing job describing the pattern and includes a brief video showcasing how to use the pattern.

Meiosis was inspired by similar projects striving for unidirectional flow of data to make state management easy like [SAM Pattern](http://sam.js.org/), [Elm Architecture](https://guide.elm-lang.org/architecture/index.html), [CycleJS](http://www.christianalfoni.com/articles/2016_04_06_CycleJS-driven-by-state), and others.

The whole Meiosis pattern is contained in the following code:

```javascript
import * as ms from 'mithril/stream'

class AppModel {
 counter: number = 0
}
type ModelUpdateFunction = (model: AppModel) => AppModel
type UpdateStream = ms.Stream<ModelUpdateFunction>

const createApp = (update: UpdateStream) : MeiosisApp => ({ 
 initialModel: ..., view: ..., render: ...
})

const setupMeiosis = (
 createApp: (s: UpdateStream) => MeiosisApp,
 container: Element
) => {
 const update = ms<ModelUpdateFunction>()
 const modelUpdate = (model: AppModel, fn: ModelUpdateFunction) => fn(model)

 const app = createApp(update)

 const models = ms.scan(modelUpdate, app.initialModel(), update)
 models.map(model => app.render(container, app.view(model)))
}

setupMeiosis(createApp, document.body.querySelector('#app') as Element)
```

Let's examine each section separately to fully understand the pattern.

```javascript
const update = ms<ModelUpdateFunction>()
```

The code above creates the `update` stream that has functions as its values. These functions receive an `AppModel` and return a new `AppModel`. These functions are going to be our actions, that are triggered either by a user interaction or by an asynchronous task, fetch, etc.

```javascript
const app = createApp(update)
```

This creates our application, which receives the `update` stream that will be used by the application's actions.

In addition, the `app` exposes three methods:

* `initialModel()` returns the initial `AppModel` which will act as the bootstrapping model for the application.
* `view(model)` receives the latest `AppModel` and returns the view to be rendered. This view can be of any framework (e.g. React, Mithril).
* `render(element, view)` receives the application view, and the DOM element to which the view will be rendered. Meiosis does not care about the framework used as long as there is functionality to render a view on-demand. Mithril, React, Preact, Inferno, etc. are able to do this efficiently with the use of virtual dom.

```javascript
const models = ms.scan(modelUpdate, app.initialModel(), update)
models.map(model => app.render(container, app.view(model)))
```

This is the juicy part of the pattern.

The `models` stream is a scanned stream over the `update` stream. This means that any function passed into the `update` stream, due to the `scan` functionality described previously, will be used to transform the current application model into a new model, and then set that model as value in the `models` stream.

The second line applies a `map` operation over the `models` stream. This will render the application view using the updated application model every time there is a new value pushed to the `update` stream. Remember that the values of the `update` stream are functions that transform the model.

This deviates slightly from traditional frameworks in the sense that when an action occurs, instead of publishing the updated model itself, the transformation functions are pushed, and the `Stream.scan` functionality takes care of applying those transformations to the model/state.

## Demo

The following code is a complete demo implementing a counter, with `plus/minus` buttons.

```javascript
import * as m from 'mithril'
import * as ms from 'mithril/stream'

class AppModel {
 counter: number = 0
}
type ModelUpdateFunction = (model: AppModel) => AppModel
type UpdateStream = ms.Stream<ModelUpdateFunction>

interface MeiosisApp {
 model: () => AppModel
 view: (model: AppModel) => m.Vnode
 render: (el: Element, v: m.Vnode) => void
}

const createActions = (update: UpdateStream) => ({
 inc: (value: number) => update(model => ({counter: model.counter + value}))
})

const createApp = (update: UpdateStream): MeiosisApp => {
 const actions = createActions(update)
 return {
 model: () => ({counter: 0}),
 view: (model: AppModel) => m("div", [
 m('p', model.counter),
 m('button', {onclick: () => actions.inc(1)}, 'plus'),
 m('button', {onclick: () => actions.inc(-1)}, 'minus')
 ]),
 render: (container: Element, v: m.Vnode) => m.render(container, v)
 }
}

const setupMeiosis = (
 createApp: (s: UpdateStream) => MeiosisApp,
 container: Element
) => {
 const update = ms<ModelUpdateFunction>()
 const modelUpdate = (model: AppModel, fn: ModelUpdateFunction) => fn(model)

 const app = createApp(update)

 const models = ms.scan(modelUpdate, app.model(), update)
 models.map(model => app.render(container, app.view(model)))
}

setupMeiosis(createApp, document.body.querySelector('#app') as Element)
```

[Demo repository on Github](https://github.com/lambrospetrou/code-playground/tree/master/meiosis-mithril-ts)

## Conclusion

After working with most of the Javascript frameworks the last few years, I enjoy having the following while developing an application:

* Immutability of state/data
* Unidirectional flow of data (actions => update state => render updated view => actions)
* Fast component rendering (the best solutions as of now use virtual dom diffing)

Adopting Meiosis pattern in my applications, means that all of the above are possible, and the only thing tied to a specific framework is really just the view components. Taking the above code, and replacing Mithril with React is just a matter of a few minutes.

I hope that more engineers will start adopting patterns that are applicable to all kinds of Javascript applications instead of focusing on framework specific solutions.

<h2 id="problem">Problem</h2>
After using <a href="https://reactjs.org/">ReactJS</a> for two years, along with all its ecosystem (<a href="https://redux.js.org/">Redux</a> for state management, <a href="https://redux-saga.js.org/">Redux-saga</a> for managing side-effects, etc.) I wanted to experiment with a simpler approach where I was not forced to use many framework specific libraries, but at the same time having the same uni-directional flow of data and the ease of UI creation using components.
A few months ago, I came across <a href="https://mithril.js.org/">Mithril</a>, a lightweight component based framework which uses pure Javascript objects for its components. I was instantly hooked and wanted to use it in all my personal projects.
How can I get the state management I want without being tied to a framework so that I can use Mithril or React interchangeably without issues?
TL;DR
My answer is using <a href="https://meiosis.js.org/">Meiosis pattern</a>.
<h2 id="streams">Streams</h2>
In order to properly understand the solution let’s go over some basics regarding streams.
There are several Javascript stream libraries but for the rest of the article I will use <a href="https://mithril.js.org/stream.html">Mithril/stream</a>.
Meiosis requires three basic operations on streams:
<ol>
<li><a href="https://mithril.js.org/stream.html#streams-as-variables">Updating the stream value</a></li>
<li><a href="https://mithril.js.org/stream.html#streammap">Mapping over the stream</a> in order to listen for updates to the stream’s value</li>
<li><a href="https://mithril.js.org/stream.html#streamscan">Scanning over the stream</a> in order to calculate accumulated value after every update</li>
</ol>
<h3 id="updating-the-stream-value">Updating the stream value</h3>
<pre><code class="language-javascript">import * as ms from &#39;mithril/stream&#39;

const s = ms()
s(1)
console.log(s()) // 1
s(s()+10)
console.log(s()) // 11</code></pre>
The example above shows that just calling a stream as a function will return the latest value of the stream, and supplying a value to the function application will update the stream’s value.
Also, the fact that the stream itself is a function can be super useful since it can be composed into more complex functions.
<h3 id="streammap">Stream.map</h3>
<pre><code class="language-javascript">import * as ms from &#39;mithril/stream&#39;

const s = ms()

s.map(value =&gt; console.log(`updated value: ${value}`))

s(1) // prints &quot;updated value: 1&quot;
s(20) // prints &quot;updated value: 20&quot;
s(311) // prints &quot;updated value: 311&quot;</code></pre>
The <a href="https://mithril.js.org/stream.html#streammap">Stream.map</a> operation is simple. It just executes a given function every time the stream value is updated, and also returns a new stream. The return value of the given function after every execution becomes the new value for the stream returned by the call to <code>map()</code>.
<h3 id="streamscan">Stream.scan</h3>
<pre><code class="language-javascript">import * as ms from &#39;mithril/stream&#39;

const accumulate = (accumulator, value) =&gt; accumulator * value
const processValue = value =&gt; console.log(value)

const s = ms()
const scanned = ms.scan(accumulate, 1, s)
scanned.map(processValue)

s(1) // prints 1
s(2) // prints 2
s(5) // prints 10
s(100) // prints 1000
s(0.5) // prints 500</code></pre>
The distilled functionality of <a href="https://mithril.js.org/stream.html#streamscan">Stream.scan</a> is that whenever the value of the source stream (<code>s</code>) changes, the <code>accumulate</code> function is called being passed the current accumulator value and the new source stream value. The return value will become the new value of the <code>scanned</code> stream and the accumulator value being passed to the next <code>accumulate</code> invocation.
The fascinating idea here is that a stream can have functions as its values.
In the following example we pass functions as values and as a result we can apply different operations on the accumulator value each time.
<pre><code class="language-javascript">import * as ms from &#39;mithril/stream&#39;

const update = (accumulator, fn) =&gt; fn(accumulator)
const processValue = value =&gt; console.log(value)

const s = ms()
const scanned = ms.scan(update, 1, s)
scanned.map(processValue)

s(value =&gt; value + 1) // prints 2
s(value =&gt; value - 1) // prints 1
s(value =&gt; value * 10) // prints 10
s(value =&gt; 1234) // prints 1234
s(value =&gt; value / 2) // prints 617</code></pre>
Spend a few minutes to fully understand the above example because it’s the bread and butter of the Meiosis pattern.
Notice how we pass functions in the stream and by the use of <code>scan</code> they are applied in order transforming the stream accumulated value.
<h2 id="meiosis-pattern">Meiosis Pattern</h2>
The <a href="https://meiosis.js.org/">Meiosis website</a> is doing an amazing job describing the pattern and includes a brief video showcasing how to use the pattern.
Meiosis was inspired by similar projects striving for unidirectional flow of data to make state management easy like <a href="http://sam.js.org/">SAM Pattern</a>, <a href="https://guide.elm-lang.org/architecture/index.html">Elm Architecture</a>, <a href="http://www.christianalfoni.com/articles/2016_04_06_CycleJS-driven-by-state">CycleJS</a>, and others.
The whole Meiosis pattern is contained in the following code:
<pre><code class="language-javascript">import * as ms from &#39;mithril/stream&#39;

class AppModel {
 counter: number = 0
}
type ModelUpdateFunction = (model: AppModel) =&gt; AppModel
type UpdateStream = ms.Stream&lt;ModelUpdateFunction&gt;

const createApp = (update: UpdateStream) : MeiosisApp =&gt; ({ 
 initialModel: ..., view: ..., render: ...
})

const setupMeiosis = (
 createApp: (s: UpdateStream) =&gt; MeiosisApp,
 container: Element
) =&gt; {
 const update = ms&lt;ModelUpdateFunction&gt;()
 const modelUpdate = (model: AppModel, fn: ModelUpdateFunction) =&gt; fn(model)

 const app = createApp(update)

 const models = ms.scan(modelUpdate, app.initialModel(), update)
 models.map(model =&gt; app.render(container, app.view(model)))
}

setupMeiosis(createApp, document.body.querySelector(&#39;#app&#39;) as Element)</code></pre>
Let’s examine each section separately to fully understand the pattern.
<pre><code class="language-javascript">const update = ms&lt;ModelUpdateFunction&gt;()</code></pre>
The code above creates the <code>update</code> stream that has functions as its values. These functions receive an <code>AppModel</code> and return a new <code>AppModel</code>. These functions are going to be our actions, that are triggered either by a user interaction or by an asynchronous task, fetch, etc.
<pre><code class="language-javascript">const app = createApp(update)</code></pre>
This creates our application, which receives the <code>update</code> stream that will be used by the application’s actions.
In addition, the <code>app</code> exposes three methods:
<ul>
<li><code>initialModel()</code> returns the initial <code>AppModel</code> which will act as the bootstrapping model for the application.</li>
<li><code>view(model)</code> receives the latest <code>AppModel</code> and returns the view to be rendered. This view can be of any framework (e.g. React, Mithril).</li>
<li><code>render(element, view)</code> receives the application view, and the DOM element to which the view will be rendered. Meiosis does not care about the framework used as long as there is functionality to render a view on-demand. Mithril, React, Preact, Inferno, etc. are able to do this efficiently with the use of virtual dom.</li>
</ul>
<pre><code class="language-javascript">const models = ms.scan(modelUpdate, app.initialModel(), update)
models.map(model =&gt; app.render(container, app.view(model)))</code></pre>
This is the juicy part of the pattern.
The <code>models</code> stream is a scanned stream over the <code>update</code> stream. This means that any function passed into the <code>update</code> stream, due to the <code>scan</code> functionality described previously, will be used to transform the current application model into a new model, and then set that model as value in the <code>models</code> stream.
The second line applies a <code>map</code> operation over the <code>models</code> stream. This will render the application view using the updated application model every time there is a new value pushed to the <code>update</code> stream. Remember that the values of the <code>update</code> stream are functions that transform the model.
This deviates slightly from traditional frameworks in the sense that when an action occurs, instead of publishing the updated model itself, the transformation functions are pushed, and the <code>Stream.scan</code> functionality takes care of applying those transformations to the model/state.
<h2 id="demo">Demo</h2>
The following code is a complete demo implementing a counter, with <code>plus/minus</code> buttons.
<pre><code class="language-javascript">import * as m from &#39;mithril&#39;
import * as ms from &#39;mithril/stream&#39;

class AppModel {
 counter: number = 0
}
type ModelUpdateFunction = (model: AppModel) =&gt; AppModel
type UpdateStream = ms.Stream&lt;ModelUpdateFunction&gt;

interface MeiosisApp {
 model: () =&gt; AppModel
 view: (model: AppModel) =&gt; m.Vnode
 render: (el: Element, v: m.Vnode) =&gt; void
}

const createActions = (update: UpdateStream) =&gt; ({
 inc: (value: number) =&gt; update(model =&gt; ({counter: model.counter + value}))
})

const createApp = (update: UpdateStream): MeiosisApp =&gt; {
 const actions = createActions(update)
 return {
 model: () =&gt; ({counter: 0}),
 view: (model: AppModel) =&gt; m(&quot;div&quot;, [
 m(&#39;p&#39;, model.counter),
 m(&#39;button&#39;, {onclick: () =&gt; actions.inc(1)}, &#39;plus&#39;),
 m(&#39;button&#39;, {onclick: () =&gt; actions.inc(-1)}, &#39;minus&#39;)
 ]),
 render: (container: Element, v: m.Vnode) =&gt; m.render(container, v)
 }
}

const setupMeiosis = (
 createApp: (s: UpdateStream) =&gt; MeiosisApp,
 container: Element
) =&gt; {
 const update = ms&lt;ModelUpdateFunction&gt;()
 const modelUpdate = (model: AppModel, fn: ModelUpdateFunction) =&gt; fn(model)

 const app = createApp(update)

 const models = ms.scan(modelUpdate, app.model(), update)
 models.map(model =&gt; app.render(container, app.view(model)))
}

setupMeiosis(createApp, document.body.querySelector(&#39;#app&#39;) as Element)</code></pre>
<a href="https://github.com/lambrospetrou/code-playground/tree/master/meiosis-mithril-ts">Demo repository on Github</a>
<h2 id="conclusion">Conclusion</h2>
After working with most of the Javascript frameworks the last few years, I enjoy having the following while developing an application:
<ul>
<li>Immutability of state/data</li>
<li>Unidirectional flow of data (actions =&gt; update state =&gt; render updated view =&gt; actions)</li>
<li>Fast component rendering (the best solutions as of now use virtual dom diffing)</li>
</ul>
Adopting Meiosis pattern in my applications, means that all of the above are possible, and the only thing tied to a specific framework is really just the view components. Taking the above code, and replacing Mithril with React is just a matter of a few minutes.
I hope that more engineers will start adopting patterns that are applicable to all kinds of Javascript applications instead of focusing on framework specific solutions.

Develop website applications without being tied to a single framework. Use the Meiosis pattern and streams to provide sane state management to your applications. Usable with Mithril, React, Preact, Inferno, and others.

Meiosis pattern - Unidirectional data flow web applications using streams


## Problem

Many times I want to run multiple services on the same EC2 instance. Sometimes I am doing a toy project and I don't want to pay for resources just hosting each project on its own, and other times, even in production systems, I need to deploy multiple microservices on the same instance, e.g. Nginx proxy, the application web service, and maybe some other monitoring service.

I am a huge fan of [AWS Elastic Beanstalk](https://aws.amazon.com/elasticbeanstalk/) and I will explain how we can achieve multi-service on same instance setup using the [Go Platform](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-environment.html) and the more flexible [Multicontainer Docker Platform](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_ecs.html).

### Desired result

The example application I will deploy has two Go web services running which just serve some static HTML and an Nginx proxy doing the routing between them.

**Web service 2** handles any request under the `/web2` path, and **Web Service 1** handles everything else.

```bash
> curl http://multiplegoservices-env.mzkfjw36fh.eu-west-1.elasticbeanstalk.com/web2
Service 2 Path, "/web2"

> curl http://multiplegoservices-env.mzkfjw36fh.eu-west-1.elasticbeanstalk.com/web1
Service 1 Path, "/web1"

> curl http://multiplegoservices-env.mzkfjw36fh.eu-west-1.elasticbeanstalk.com/
Service 1 Path, "/"
```

The code for our services is **exactly the same** between the two platforms, and the only difference is among the files specific for each platform configuration.

## Go Platform

The Go Platform in Elastic Beanstalk is pretty simple, with the most important concepts being the [Procfile](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-procfile.html), the [Buildfile](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-buildfile.html), and the special `.ebextensions` folder which we will use to provide [custom configuration to the Nginx proxy](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-nginx.html) deployed automatically on the instances.

[Source code available on Github](https://github.com/lambrospetrou/aws-playground/tree/master/elastic-beanstalk-multiple-applications).

### Procfile

This file contains a simple enumeration of the services to start (i.e. executables to run). For the example application, `Procfile` contains the following:

```
web_service1: bin/web-service-1
web_service2: bin/web-service-2
```

One thing you need to know is that Elastic Beanstalk will start the first service setting the environment variable `PORT=5000`, and that is the port your service should listen for requests. Each subsequent service will receive `PORT` values in **100 increments** from the last one, i.e. `web-service-2` will have `PORT=5100`.

### Buildfile

This file contains a simple enumeration of commands to run during the deployment artifact build time. This can be used to build your code and generate the executables that `Procfile` will execute, but in my case I like building locally (or in a pipeline) and just deploy the executables as the Elastic Beanstalk artifact.

Just for the sake of using `Buildfile`, I call a bash script that prints `Hello world!`, and it looks like below:

```
command_to_run_during_build: bin/hello.sh
```

As a general guideline, the `Buildfile` can be used for any arbitrary task that needs to run before the services are started.

### Nginx proxy configuration

For the proxy configuration we just need a `server {}` Nginx directive to provide the routing between the two services. To achieve this, we create a **.conf** file inside the `.ebextensions/nginx/conf.d/` directory which will be included by Nginx during startup.

The following configuration is enough to do the job, and is in the file `.ebextensions/nginx/conf.d/01_proxy.conf`.

```
server {
    server_name .elasticbeanstalk.com;
    listen 80;

    location /web2 {
        proxy_pass http://127.0.0.1:5100;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location / {
        proxy_pass http://127.0.0.1:5000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
```

### Deployment artifact

All the paths used in the files above are based on the following artifact (the `.zip` file) I deploy to Elastic Beanstalk. Full listing of the contents below:

```bash
> unzip -l build/bundle.zip 
Archive:  build/bundle.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
       64  2018-03-11 13:44   Procfile
        0  2018-03-11 13:44   .ebextensions/
        0  2018-03-11 13:44   .ebextensions/nginx/
        0  2018-03-11 13:44   .ebextensions/nginx/conf.d/
      319  2018-03-11 13:44   .ebextensions/nginx/conf.d/01_proxy.conf
        0  2018-03-11 13:44   bin/
  6218916  2018-03-11 13:44   bin/web-service-1
       38  2018-03-11 13:44   bin/hello.sh
  6218916  2018-03-11 13:44   bin/web-service-2
       42  2018-03-11 13:44   Buildfile
---------                     -------
 12438295                     10 files
```

## Multicontainer Docker Platform

The multicontainer Docker platform uses [Amazon Elastic Container Service](https://aws.amazon.com/ecs/) under the covers, but as I said before, deploying through Elastic Beanstalk makes things a lot easier!

There is only one important configuration file in the Multicontainer Docker platform and that is the [Dockerrun.aws.json](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_v2config.html).

This file contains Docker specific definitions, e.g. the Docker images we want for each service, the volume definitions mapping to source paths in our deployment artifact, etc.

[Source code available on Github](https://github.com/lambrospetrou/aws-playground/tree/master/elastic-beanstalk-multicontainer-docker).

### Dockerrun.aws.json

In order to achieve the same result as with the **Go Platform** I use the following `Dockerrun.aws.json` file.

```json
{
  "AWSEBDockerrunVersion": 2,
  "volumes": [
    {
      "name": "web1",
      "host": {
        "sourcePath": "/var/app/current/web-service-1"
      }
    },
    {
      "name": "web2",
      "host": {
        "sourcePath": "/var/app/current/web-service-2"
      }
    },
    {
      "name": "nginx-proxy-conf",
      "host": {
        "sourcePath": "/var/app/current/proxy/conf.d"
      }
    }
  ],
  "containerDefinitions": [
    {
      "name": "web1",
      "image": "golang:1.10",
      "essential": true,
      "memory": 128,
      "mountPoints": [
        {
          "sourceVolume": "web1",
          "containerPath": "/var/app"
        }
      ],
      "portMappings": [
        {
          "hostPort": 5000,
          "containerPort": 5000
        }
      ],
      "environment": [
        {
          "name": "PORT",
          "value": "5000"
        }
      ],
      "command": ["/var/app/web-service-1"]
    },
    {
      "name": "web2",
      "image": "golang:1.10",
      "essential": true,
      "memory": 128,
      "mountPoints": [
        {
          "sourceVolume": "web2",
          "containerPath": "/var/app"
        }
      ],
      "portMappings": [
        {
          "hostPort": 5100,
          "containerPort": 5100
        }
      ],
      "environment": [
        {
          "name": "PORT",
          "value": "5100"
        }
      ],
      "command": ["/var/app/web-service-2"]
    },
    {
      "name": "nginx-proxy",
      "image": "nginx",
      "essential": true,
      "memory": 128,
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80
        }
      ],
      "links": [
        "web1", "web2"
      ],
      "mountPoints": [
        {
          "sourceVolume": "nginx-proxy-conf",
          "containerPath": "/etc/nginx/conf.d"
        },
        {
          "sourceVolume": "awseb-logs-nginx-proxy",
          "containerPath": "/var/log/nginx"
        }
      ]
    }
  ]
}
```

**Notes**

* `/var/app/current` is the directory on the host machine that contains our deployment artifact, i.e. the `.zip` file unzipped.
* In order to allow the Nginx image to communicate with the two services running we need to **link** those images to the `nginx-proxy` image, and instead of using `http://127.0.0.1` in the Nginx `.conf` file we should use `http://web1` and `http://web2` as shown below.

    ```
    server {
        server_name .elasticbeanstalk.com;
        listen 80;

        location /web2 {
            proxy_pass http://web2:5100;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        location / {
            proxy_pass http://web1:5000;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
    ```

### Deployment artifact

Full listing of the deployment artifact contents is as follows:

```bash
> unzip -l build/bundle.zip 
Archive:  build/bundle.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
     1960  2018-03-10 19:01   Dockerrun.aws.json
        0  2018-03-10 19:01   proxy/
        0  2018-03-10 19:01   proxy/conf.d/
      326  2018-03-10 19:01   proxy/conf.d/default.conf
        0  2018-03-10 19:01   web-service-1/
  6218916  2018-03-10 19:01   web-service-1/web-service-1
        0  2018-03-10 19:01   web-service-2/
  6218916  2018-03-10 19:01   web-service-2/web-service-2
---------                     -------
 12440118                     8 files
``` 

## Conclusion

AWS Elastic Beanstalk is an amazing service which abstracts a lot of stuff that are not really part of the application, e.g. load balancers, autoscaling groups, logging, alarms, and there is even a super helpful dashboard right out-of-the box.

My simplistic guideline on what platform to use is as follows:

**Go Platform**

* Your services are written in Go
* Your services can be compiled into binary executables that run on Amazon Linux

**Multicontainer Docker Platform**

* Anything else

Have fun microservicing with AWS Elastic Beanstalk!


<h2 id="problem">Problem</h2>
Many times I want to run multiple services on the same EC2 instance. Sometimes I am doing a toy project and I don’t want to pay for resources just hosting each project on its own, and other times, even in production systems, I need to deploy multiple microservices on the same instance, e.g. Nginx proxy, the application web service, and maybe some other monitoring service.
I am a huge fan of <a href="https://aws.amazon.com/elasticbeanstalk/">AWS Elastic Beanstalk</a> and I will explain how we can achieve multi-service on same instance setup using the <a href="https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-environment.html">Go Platform</a> and the more flexible <a href="https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_ecs.html">Multicontainer Docker Platform</a>.
<h3 id="desired-result">Desired result</h3>
The example application I will deploy has two Go web services running which just serve some static HTML and an Nginx proxy doing the routing between them.
Web service 2 handles any request under the <code>/web2</code> path, and Web Service 1 handles everything else.
<pre><code class="language-bash">&gt; curl http://multiplegoservices-env.mzkfjw36fh.eu-west-1.elasticbeanstalk.com/web2
Service 2 Path, &quot;/web2&quot;

&gt; curl http://multiplegoservices-env.mzkfjw36fh.eu-west-1.elasticbeanstalk.com/web1
Service 1 Path, &quot;/web1&quot;

&gt; curl http://multiplegoservices-env.mzkfjw36fh.eu-west-1.elasticbeanstalk.com/
Service 1 Path, &quot;/&quot;</code></pre>
The code for our services is exactly the same between the two platforms, and the only difference is among the files specific for each platform configuration.
<h2 id="go-platform">Go Platform</h2>
The Go Platform in Elastic Beanstalk is pretty simple, with the most important concepts being the <a href="https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-procfile.html">Procfile</a>, the <a href="https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-buildfile.html">Buildfile</a>, and the special <code>.ebextensions</code> folder which we will use to provide <a href="https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-nginx.html">custom configuration to the Nginx proxy</a> deployed automatically on the instances.
<a href="https://github.com/lambrospetrou/aws-playground/tree/master/elastic-beanstalk-multiple-applications">Source code available on Github</a>.
<h3 id="procfile">Procfile</h3>
This file contains a simple enumeration of the services to start (i.e. executables to run). For the example application, <code>Procfile</code> contains the following:
<pre><code>web_service1: bin/web-service-1
web_service2: bin/web-service-2</code></pre>One thing you need to know is that Elastic Beanstalk will start the first service setting the environment variable <code>PORT=5000</code>, and that is the port your service should listen for requests. Each subsequent service will receive <code>PORT</code> values in 100 increments from the last one, i.e. <code>web-service-2</code> will have <code>PORT=5100</code>.
<h3 id="buildfile">Buildfile</h3>
This file contains a simple enumeration of commands to run during the deployment artifact build time. This can be used to build your code and generate the executables that <code>Procfile</code> will execute, but in my case I like building locally (or in a pipeline) and just deploy the executables as the Elastic Beanstalk artifact.
Just for the sake of using <code>Buildfile</code>, I call a bash script that prints <code>Hello world!</code>, and it looks like below:
<pre><code>command_to_run_during_build: bin/hello.sh</code></pre>As a general guideline, the <code>Buildfile</code> can be used for any arbitrary task that needs to run before the services are started.
<h3 id="nginx-proxy-configuration">Nginx proxy configuration</h3>
For the proxy configuration we just need a <code>server {}</code> Nginx directive to provide the routing between the two services. To achieve this, we create a .conf file inside the <code>.ebextensions/nginx/conf.d/</code> directory which will be included by Nginx during startup.
The following configuration is enough to do the job, and is in the file <code>.ebextensions/nginx/conf.d/01_proxy.conf</code>.
<pre><code>server {
 server_name .elasticbeanstalk.com;
 listen 80;

 location /web2 {
 proxy_pass http://127.0.0.1:5100;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 }

 location / {
 proxy_pass http://127.0.0.1:5000;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 }
}</code></pre><h3 id="deployment-artifact">Deployment artifact</h3>
All the paths used in the files above are based on the following artifact (the <code>.zip</code> file) I deploy to Elastic Beanstalk. Full listing of the contents below:
<pre><code class="language-bash">&gt; unzip -l build/bundle.zip 
Archive: build/bundle.zip
 Length Date Time Name
--------- ---------- ----- ----
 64 2018-03-11 13:44 Procfile
 0 2018-03-11 13:44 .ebextensions/
 0 2018-03-11 13:44 .ebextensions/nginx/
 0 2018-03-11 13:44 .ebextensions/nginx/conf.d/
 319 2018-03-11 13:44 .ebextensions/nginx/conf.d/01_proxy.conf
 0 2018-03-11 13:44 bin/
 6218916 2018-03-11 13:44 bin/web-service-1
 38 2018-03-11 13:44 bin/hello.sh
 6218916 2018-03-11 13:44 bin/web-service-2
 42 2018-03-11 13:44 Buildfile
--------- -------
 12438295 10 files</code></pre>
<h2 id="multicontainer-docker-platform">Multicontainer Docker Platform</h2>
The multicontainer Docker platform uses <a href="https://aws.amazon.com/ecs/">Amazon Elastic Container Service</a> under the covers, but as I said before, deploying through Elastic Beanstalk makes things a lot easier!
There is only one important configuration file in the Multicontainer Docker platform and that is the <a href="https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_v2config.html">Dockerrun.aws.json</a>.
This file contains Docker specific definitions, e.g. the Docker images we want for each service, the volume definitions mapping to source paths in our deployment artifact, etc.
<a href="https://github.com/lambrospetrou/aws-playground/tree/master/elastic-beanstalk-multicontainer-docker">Source code available on Github</a>.
<h3 id="dockerrunawsjson">Dockerrun.aws.json</h3>
In order to achieve the same result as with the Go Platform I use the following <code>Dockerrun.aws.json</code> file.
<pre><code class="language-json">{
 &quot;AWSEBDockerrunVersion&quot;: 2,
 &quot;volumes&quot;: [
 {
 &quot;name&quot;: &quot;web1&quot;,
 &quot;host&quot;: {
 &quot;sourcePath&quot;: &quot;/var/app/current/web-service-1&quot;
 }
 },
 {
 &quot;name&quot;: &quot;web2&quot;,
 &quot;host&quot;: {
 &quot;sourcePath&quot;: &quot;/var/app/current/web-service-2&quot;
 }
 },
 {
 &quot;name&quot;: &quot;nginx-proxy-conf&quot;,
 &quot;host&quot;: {
 &quot;sourcePath&quot;: &quot;/var/app/current/proxy/conf.d&quot;
 }
 }
 ],
 &quot;containerDefinitions&quot;: [
 {
 &quot;name&quot;: &quot;web1&quot;,
 &quot;image&quot;: &quot;golang:1.10&quot;,
 &quot;essential&quot;: true,
 &quot;memory&quot;: 128,
 &quot;mountPoints&quot;: [
 {
 &quot;sourceVolume&quot;: &quot;web1&quot;,
 &quot;containerPath&quot;: &quot;/var/app&quot;
 }
 ],
 &quot;portMappings&quot;: [
 {
 &quot;hostPort&quot;: 5000,
 &quot;containerPort&quot;: 5000
 }
 ],
 &quot;environment&quot;: [
 {
 &quot;name&quot;: &quot;PORT&quot;,
 &quot;value&quot;: &quot;5000&quot;
 }
 ],
 &quot;command&quot;: [&quot;/var/app/web-service-1&quot;]
 },
 {
 &quot;name&quot;: &quot;web2&quot;,
 &quot;image&quot;: &quot;golang:1.10&quot;,
 &quot;essential&quot;: true,
 &quot;memory&quot;: 128,
 &quot;mountPoints&quot;: [
 {
 &quot;sourceVolume&quot;: &quot;web2&quot;,
 &quot;containerPath&quot;: &quot;/var/app&quot;
 }
 ],
 &quot;portMappings&quot;: [
 {
 &quot;hostPort&quot;: 5100,
 &quot;containerPort&quot;: 5100
 }
 ],
 &quot;environment&quot;: [
 {
 &quot;name&quot;: &quot;PORT&quot;,
 &quot;value&quot;: &quot;5100&quot;
 }
 ],
 &quot;command&quot;: [&quot;/var/app/web-service-2&quot;]
 },
 {
 &quot;name&quot;: &quot;nginx-proxy&quot;,
 &quot;image&quot;: &quot;nginx&quot;,
 &quot;essential&quot;: true,
 &quot;memory&quot;: 128,
 &quot;portMappings&quot;: [
 {
 &quot;hostPort&quot;: 80,
 &quot;containerPort&quot;: 80
 }
 ],
 &quot;links&quot;: [
 &quot;web1&quot;, &quot;web2&quot;
 ],
 &quot;mountPoints&quot;: [
 {
 &quot;sourceVolume&quot;: &quot;nginx-proxy-conf&quot;,
 &quot;containerPath&quot;: &quot;/etc/nginx/conf.d&quot;
 },
 {
 &quot;sourceVolume&quot;: &quot;awseb-logs-nginx-proxy&quot;,
 &quot;containerPath&quot;: &quot;/var/log/nginx&quot;
 }
 ]
 }
 ]
}</code></pre>
Notes
<ul>
<li><code>/var/app/current</code> is the directory on the host machine that contains our deployment artifact, i.e. the <code>.zip</code> file unzipped.
</li>
<li>In order to allow the Nginx image to communicate with the two services running we need to link those images to the <code>nginx-proxy</code> image, and instead of using <code>http://127.0.0.1</code> in the Nginx <code>.conf</code> file we should use <code>http://web1</code> and <code>http://web2</code> as shown below.
<pre><code> server {
 server_name .elasticbeanstalk.com;
 listen 80;

 location /web2 {
 proxy_pass http://web2:5100;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 }

 location / {
 proxy_pass http://web1:5000;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 }
 }</code></pre></li>
</ul>
<h3 id="deployment-artifact-1">Deployment artifact</h3>
Full listing of the deployment artifact contents is as follows:
<pre><code class="language-bash">&gt; unzip -l build/bundle.zip 
Archive: build/bundle.zip
 Length Date Time Name
--------- ---------- ----- ----
 1960 2018-03-10 19:01 Dockerrun.aws.json
 0 2018-03-10 19:01 proxy/
 0 2018-03-10 19:01 proxy/conf.d/
 326 2018-03-10 19:01 proxy/conf.d/default.conf
 0 2018-03-10 19:01 web-service-1/
 6218916 2018-03-10 19:01 web-service-1/web-service-1
 0 2018-03-10 19:01 web-service-2/
 6218916 2018-03-10 19:01 web-service-2/web-service-2
--------- -------
 12440118 8 files</code></pre>
<h2 id="conclusion">Conclusion</h2>
AWS Elastic Beanstalk is an amazing service which abstracts a lot of stuff that are not really part of the application, e.g. load balancers, autoscaling groups, logging, alarms, and there is even a super helpful dashboard right out-of-the box.
My simplistic guideline on what platform to use is as follows:
Go Platform
<ul>
<li>Your services are written in Go</li>
<li>Your services can be compiled into binary executables that run on Amazon Linux</li>
</ul>
Multicontainer Docker Platform
<ul>
<li>Anything else</li>
</ul>
Have fun microservicing with AWS Elastic Beanstalk!

A tutorial explaining how to run multiple services on the same instance using AWS Elastic Beanstalk. The Go and Multicontainer platforms are examined.

Run multiple services on a single EC2 instance using AWS Elastic Beanstalk (Go and Multicontainer Docker platforms)


## Problem

I have some important private files that I want to store in [Google Drive](https://www.google.com/drive/) and on my USB flash drive, but I don't want them to be in plain sight for anyone to see.

I would like to at least password-protect them before storing them, but without too much hassle with asymmetric cryptography where I need to fiddle with keys.

## Solution

It turns out pretty much all UNIX systems have [GnuPG](https://www.gnupg.org/) installed which allows me to just run a command to encrypt a file using a passphrase, and a corresponding command to decrypt it when I need to open it.

I found out that this method is also used [inside NASA when transferring files](https://www.nas.nasa.gov/hecc/support/kb/using-gpg-to-encrypt-your-data_242.html).

In order to **encrypt and password-protect a file** run the following command:

```bash
gpg -c --cipher-algo AES256 private-file.txt
```

The `-c` option specifies that we want to do symmetric encryption using a passphrase. The `--cipher-algo 256` option specifies that we want to use the [AES256 cipher](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) instead of the default [CAST5 cipher](https://en.wikipedia.org/wiki/CAST-128), although this is not required.

The above command will ask you for the passphrase to use, and then will create a new file named `private-file.txt.gpg`, which is the encrypted and password-protected file we want to store.

In order to **decrypt the file** run the following command:

```bash
gpg private-file.txt.gpg
```

Once you enter the passphrase used during the encryption of the file, you will get back the decrypted file which will have the same name without the `.gpg` extension, hence `private-file.txt`.

**Tips**

* If you want to encrypt a whole directory (folder), then you have to first zip/tar the folder into a single file and then apply the same command above to the zipped/tarred file.
* Use long passphrases for important files consisting of multiple words with letters, spaces, symbols, and numbers to maximise the entropy and security of the encryption.

## References

* [NASA - Using GPG to Encrypt Your Data](https://www.nas.nasa.gov/hecc/support/kb/using-gpg-to-encrypt-your-data_242.html)
* [How to use a password stored in separate file as passphrase](https://www.cyberciti.biz/tips/linux-how-to-encrypt-and-decrypt-files-with-a-password.html#comment-4392)


<h2 id="problem">Problem</h2>
I have some important private files that I want to store in <a href="https://www.google.com/drive/">Google Drive</a> and on my USB flash drive, but I don’t want them to be in plain sight for anyone to see.
I would like to at least password-protect them before storing them, but without too much hassle with asymmetric cryptography where I need to fiddle with keys.
<h2 id="solution">Solution</h2>
It turns out pretty much all UNIX systems have <a href="https://www.gnupg.org/">GnuPG</a> installed which allows me to just run a command to encrypt a file using a passphrase, and a corresponding command to decrypt it when I need to open it.
I found out that this method is also used <a href="https://www.nas.nasa.gov/hecc/support/kb/using-gpg-to-encrypt-your-data_242.html">inside NASA when transferring files</a>.
In order to encrypt and password-protect a file run the following command:
<pre><code class="language-bash">gpg -c --cipher-algo AES256 private-file.txt</code></pre>
The <code>-c</code> option specifies that we want to do symmetric encryption using a passphrase. The <code>--cipher-algo 256</code> option specifies that we want to use the <a href="https://en.wikipedia.org/wiki/Advanced_Encryption_Standard">AES256 cipher</a> instead of the default <a href="https://en.wikipedia.org/wiki/CAST-128">CAST5 cipher</a>, although this is not required.
The above command will ask you for the passphrase to use, and then will create a new file named <code>private-file.txt.gpg</code>, which is the encrypted and password-protected file we want to store.
In order to decrypt the file run the following command:
<pre><code class="language-bash">gpg private-file.txt.gpg</code></pre>
Once you enter the passphrase used during the encryption of the file, you will get back the decrypted file which will have the same name without the <code>.gpg</code> extension, hence <code>private-file.txt</code>.
Tips
<ul>
<li>If you want to encrypt a whole directory (folder), then you have to first zip/tar the folder into a single file and then apply the same command above to the zipped/tarred file.</li>
<li>Use long passphrases for important files consisting of multiple words with letters, spaces, symbols, and numbers to maximise the entropy and security of the encryption.</li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><a href="https://www.nas.nasa.gov/hecc/support/kb/using-gpg-to-encrypt-your-data_242.html">NASA - Using GPG to Encrypt Your Data</a></li>
<li><a href="https://www.cyberciti.biz/tips/linux-how-to-encrypt-and-decrypt-files-with-a-password.html#comment-4392">How to use a password stored in separate file as passphrase</a></li>
</ul>

An easy way to encrypt and password-protect a file on a UNIX OS.

Encrypt files with password on Linux

## Problem

Recently, I have updated the [deployment scripts of my website](https://github.com/lambrospetrou/lambrospetrou.github.io) and for some tasks I wanted to have certain environment variables available in my scripts. For example, I wanted to [invalidate the Cloudfront distribution](http://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-invalidation.html) so that I don't have to wait for the cache to expire before serving the new content, and I needed the `--distribution-id`.

I looked for the easiest way to export environment variables from key-value pairs in a file, and I was delighted to see that it can done with a single line!

## Solution

The solution is based on [this StackOverflow answer](https://stackoverflow.com/a/36456837/1066790).

Assuming the following `aws.environment` file:

```bash
DISTRIBUTION_ID=xxxxxxxxx
S3_BUCKET=www.example.com
```

The following script will export the above key-value pairs as environment variables.

```bash
#!/usr/bin/env bash

source <(sed -E -n 's/[^#]+/export &/ p' aws.environment)

# ... Commands that use the above variables i.e. echo "$DISTRIBUTION_ID"
```

In addition, I didn't want to limit myself only to one file of environment variables because I would like to version control some of them in git. So, I wanted to scan and export variables from all the files in a given directory. This can be done using the following script.

```bash
#!/usr/bin/env bash

source <(find ./build-tool/env/ -type f -exec sed -E -n 's/[^#]+/export &/ p' {} +)

# ... Commands that use the above variables
```

This final adaptation uses the [find command](https://ss64.com/bash/find.html) to read all files in the `./build-tool/env` directory and for each one exports its key-value pairs as environment variables.

That's it! Once again, knowledge of some nifty Linux commands saves the day.

<h2 id="problem">Problem</h2>
Recently, I have updated the <a href="https://github.com/lambrospetrou/lambrospetrou.github.io">deployment scripts of my website</a> and for some tasks I wanted to have certain environment variables available in my scripts. For example, I wanted to <a href="http://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-invalidation.html">invalidate the Cloudfront distribution</a> so that I don’t have to wait for the cache to expire before serving the new content, and I needed the <code>--distribution-id</code>.
I looked for the easiest way to export environment variables from key-value pairs in a file, and I was delighted to see that it can done with a single line!
<h2 id="solution">Solution</h2>
The solution is based on <a href="https://stackoverflow.com/a/36456837/1066790">this StackOverflow answer</a>.
Assuming the following <code>aws.environment</code> file:
<pre><code class="language-bash">DISTRIBUTION_ID=xxxxxxxxx
S3_BUCKET=www.example.com</code></pre>
The following script will export the above key-value pairs as environment variables.
<pre><code class="language-bash">#!/usr/bin/env bash

source &lt;(sed -E -n &#39;s/[^#]+/export &amp;/ p&#39; aws.environment)

# ... Commands that use the above variables i.e. echo &quot;$DISTRIBUTION_ID&quot;</code></pre>
In addition, I didn’t want to limit myself only to one file of environment variables because I would like to version control some of them in git. So, I wanted to scan and export variables from all the files in a given directory. This can be done using the following script.
<pre><code class="language-bash">#!/usr/bin/env bash

source &lt;(find ./build-tool/env/ -type f -exec sed -E -n &#39;s/[^#]+/export &amp;/ p&#39; {} +)

# ... Commands that use the above variables</code></pre>
This final adaptation uses the <a href="https://ss64.com/bash/find.html">find command</a> to read all files in the <code>./build-tool/env</code> directory and for each one exports its key-value pairs as environment variables.
That’s it! Once again, knowledge of some nifty Linux commands saves the day.

Easy way to export environment variables defined in multiple files inside a directory, using Bash or any other Linux shell.

Export environment variables from multiple files using Bash on Linux


I recently posted a detailed article on [how to efficiently run Racket compiled native binaries on AWS Lambda](/articles/aws-lambda-meets-racket/), and the same process can be used for any compiled binary that runs on Amazon Linux (including binaries from languages like Go, Racket, OCaml, Rust, C/C++, etc.).

The Lambda wrapper code was very easy to understand but a bit long, and if you have lots of lambdas using binaries you might end up copy-pasting those lines, which as we know is not good. Remember the [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) principle?

As a result, to make it easier for me, and anyone implementing the solution described in that article, I created an npm package to help you reduce the boilerplate of your Lambda wrapper code significantly.

## AWS-Lambda-Binary npm package

You can find the package at https://www.npmjs.com/package/aws-lambda-binary

Its usage is very easy and the package itself provides sufficient documentation of the API. In addition, couple of ready-to-upload AWS Lambda examples are provided inside the ```_examples``` folder.

In a nutshell, the following describes how to use this package to start the Linux command [cat](https://ss64.com/bash/cat.html) which will be used as an echo-back process.

First of all if you don't have an existing lambda project, just create one using the following commands:

```bash
mkdir lambda-test && cd lambda-test
npm init -y
```

Now that you have a project, install the npm package:

```bash
npm install aws-lambda-binary
```

Copy and paste the following code into a file named ```wrapper.js```.

```javascript
const application = require('aws-lambda-binary').spawnLineByLine({
    spawn: { command: 'cat' }
});

exports.handler = function (event, context) {
    application.ensureIsRunning();

    application.stdout(result => context.done(null, result));

    application.stdin(JSON.stringify({event, context}));
};
```

Finally, prepare the zip file to upload to AWS Lambda by running the command below:

```bash
zip -r bundle.zip wrapper.js node_modules/
```

In the [AWS Lambda console](https://console.aws.amazon.com/lambda/home), create a new function with **Runtime** ```Node.js 6.10``` (or higher), and with **Handler** ```wrapper.handler```.

Upload the ```bundle.zip``` file you created above and **Save and Test**. The above lamba code takes the **event** of the lambda invocation, along with the **context** object which contains some metadata, serialises them into JSON string and sends them to the ```cat``` process through standard input. The process returns the result back through standard output and we successfully finish the lambda invocation passing the line of text received as the result.

There are more examples in the npm package but as you can see above, once you have all your logic into a compiled binary, starting the binary and communicating with it is a matter of ~5 lines.

Enjoy, and feel free to contribute to the package.

SEO tags: AWS, AWSLambda, binary, racket, go, ocaml, rust


I recently posted a detailed article on <a href="/articles/aws-lambda-meets-racket/">how to efficiently run Racket compiled native binaries on AWS Lambda</a>, and the same process can be used for any compiled binary that runs on Amazon Linux (including binaries from languages like Go, Racket, OCaml, Rust, C/C++, etc.).
The Lambda wrapper code was very easy to understand but a bit long, and if you have lots of lambdas using binaries you might end up copy-pasting those lines, which as we know is not good. Remember the <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a> principle?
As a result, to make it easier for me, and anyone implementing the solution described in that article, I created an npm package to help you reduce the boilerplate of your Lambda wrapper code significantly.
<h2 id="aws-lambda-binary-npm-package">AWS-Lambda-Binary npm package</h2>
You can find the package at <a href="https://www.npmjs.com/package/aws-lambda-binary">https://www.npmjs.com/package/aws-lambda-binary</a>
Its usage is very easy and the package itself provides sufficient documentation of the API. In addition, couple of ready-to-upload AWS Lambda examples are provided inside the <code>_examples</code> folder.
In a nutshell, the following describes how to use this package to start the Linux command <a href="https://ss64.com/bash/cat.html">cat</a> which will be used as an echo-back process.
First of all if you don’t have an existing lambda project, just create one using the following commands:
<pre><code class="language-bash">mkdir lambda-test &amp;&amp; cd lambda-test
npm init -y</code></pre>
Now that you have a project, install the npm package:
<pre><code class="language-bash">npm install aws-lambda-binary</code></pre>
Copy and paste the following code into a file named <code>wrapper.js</code>.
<pre><code class="language-javascript">const application = require(&#39;aws-lambda-binary&#39;).spawnLineByLine({
 spawn: { command: &#39;cat&#39; }
});

exports.handler = function (event, context) {
 application.ensureIsRunning();

 application.stdout(result =&gt; context.done(null, result));

 application.stdin(JSON.stringify({event, context}));
};</code></pre>
Finally, prepare the zip file to upload to AWS Lambda by running the command below:
<pre><code class="language-bash">zip -r bundle.zip wrapper.js node_modules/</code></pre>
In the <a href="https://console.aws.amazon.com/lambda/home">AWS Lambda console</a>, create a new function with Runtime <code>Node.js 6.10</code> (or higher), and with Handler <code>wrapper.handler</code>.
Upload the <code>bundle.zip</code> file you created above and Save and Test. The above lamba code takes the event of the lambda invocation, along with the context object which contains some metadata, serialises them into JSON string and sends them to the <code>cat</code> process through standard input. The process returns the result back through standard output and we successfully finish the lambda invocation passing the line of text received as the result.
There are more examples in the npm package but as you can see above, once you have all your logic into a compiled binary, starting the binary and communicating with it is a matter of ~5 lines.
Enjoy, and feel free to contribute to the package.
SEO tags: AWS, AWSLambda, binary, racket, go, ocaml, rust

Brief description of my npm package that makes it easier to run compiled binaries on AWS Lambda or in your server applications.

npm package to easily run arbitrary compiled binaries in your applications or AWS Lambda


Since I have been fiddling around with [Windows Subsystem for Linux](https://msdn.microsoft.com/en-gb/commandline/wsl/install_guide) I realised how bad **Command Prompt** is compared to [Terminator](https://gnometerminator.blogspot.co.uk/p/introduction.html), my go-to terminal solution for Linux.

I wanted to find an alternative which at least offered multiple tabs, proper text selection and in general being a better terminal for Windows.

I tried several ones but for now I settled with [ConEmu](https://conemu.github.io/) which has the extra benefit of being open-source.

One problem I had with it, is that when you go into the WSL mode, using the ```bash``` command, the arrow keys do not work. This was weird and confusing at first since they work if you go into WSL mode using **Command Prompt**.

The solution is to pass an additional argument when entering the WSL mode through ConEmu, as shown below.

```bash
# `p1` refers to tab-1.
# If you are entering this command to a different tab in ConEmu
# you have to use the proper tab-id.
bash -cur_console:p1

# Or you can open a new tab straight into WSL mode and avoid the tab-id
bash -new_console
```

Now all the keys should work properly.

This nice terminal application, along with the ability to use editors straight from the Linux subsystem as described in a [previous article](/articles/windows-linux-subsystem-editor-setup/) make Windows 10 a very viable solution for developers that love the command line.

Since I have been fiddling around with <a href="https://msdn.microsoft.com/en-gb/commandline/wsl/install_guide">Windows Subsystem for Linux</a> I realised how bad Command Prompt is compared to <a href="https://gnometerminator.blogspot.co.uk/p/introduction.html">Terminator</a>, my go-to terminal solution for Linux.
I wanted to find an alternative which at least offered multiple tabs, proper text selection and in general being a better terminal for Windows.
I tried several ones but for now I settled with <a href="https://conemu.github.io/">ConEmu</a> which has the extra benefit of being open-source.
One problem I had with it, is that when you go into the WSL mode, using the <code>bash</code> command, the arrow keys do not work. This was weird and confusing at first since they work if you go into WSL mode using Command Prompt.
The solution is to pass an additional argument when entering the WSL mode through ConEmu, as shown below.
<pre><code class="language-bash"># `p1` refers to tab-1.
# If you are entering this command to a different tab in ConEmu
# you have to use the proper tab-id.
bash -cur_console:p1

# Or you can open a new tab straight into WSL mode and avoid the tab-id
bash -new_console</code></pre>
Now all the keys should work properly.
This nice terminal application, along with the ability to use editors straight from the Linux subsystem as described in a <a href="/articles/windows-linux-subsystem-editor-setup/">previous article</a> make Windows 10 a very viable solution for developers that love the command line.

I wanted a better terminal, or command prompt, for Windows so I installed ConEmu and am really happy with it.

Command prompt alternatives for Windows


## Problem

While I was testing out the [Windows Subsystem for Linux - WSL](https://msdn.microsoft.com/en-gb/commandline/wsl/install_guide) I setup all my favourite programming languages inside the Linux system instead of Windows. Everything was working smoothly until I had to use an editor to work on some projects.

I initially installed [Visual Studio Code](https://code.visualstudio.com/) in my Windows system but realised that I don't have access to the ```PATH``` or the binaries of my languages installed inside WSL. The solution is to have an X-server running in Windows and then run the editor from inside the WSL but rendering to the X-server display (easier than it sounds).

BTW, if you are a VIM user you can stop reading right now since VIM works out of the box.

## Solution

Firstly, we need to install an X-server in Windows. I installed [Xming X Server for Windows](https://sourceforge.net/projects/xming/) and it works like a charm, with all the default options. Just download, install, and run!

If the server is running successfully, you should see an icon in the notification area of the taskbar that says **Xming Server:0.0**.

Once the X-server is running we are ready to go. In order to start an application with a GUI from inside WSL and have it render in Windows we need to set the environment variable ```DISPLAY=:0``` and then run the command.

```bash
# Either set it right before the command
DISPLAY=:0 command-with-a-gui

# or set it for the whole terminal session
export DISPLAY=:0
command-with-a-gui
```

### Visual Studio Code

I started using Visual Studio Code for the most part of last year and I was really happy with it, but it seems that it cannot properly run through an X server and it is [not a priority of its team to get it working](https://github.com/Microsoft/vscode/issues/13138).

Not a huge issue since my previously favourite editor is perfectly working, but still a bummer since Microsoft's own editor does not work with their system!

### Sublime Text 3

The [Sublime Text 3 installation instructions](https://www.sublimetext.com/docs/3/linux_repositories.html) are straightforward but I copy the commands needed here to make the article more complete.

```bash
wget -qO - https://download.sublimetext.com/sublimehq-pub.gpg | sudo apt-key add -
echo "deb https://download.sublimetext.com/ apt/stable/" | sudo tee /etc/apt/sources.list.d/sublime-text.list
sudo apt-get update && sudo apt-get install sublime-text
```

The command to run Sublime Text 3 is ```subl```. So in conjunction with the X-server display variable we can run Sublime using the following command:

```bash
DISPLAY=:0 subl
```

The above command should open a window of Sublime Text inside Windows, and if you try to **Open folder** you should be seeing the Linux subsystem file-system.

The next step is to [install the package control](https://packagecontrol.io/installation) for Sublime so that we can install our favourite plugins.

## Next steps

The font I use for all my text editors, and IDEs, is [Source Code Pro](https://github.com/adobe-fonts/source-code-pro/), which in Ubuntu-like systems can be installed easily using the following commands:

```bash
git clone --depth 1 --branch release https://github.com/adobe-fonts/source-code-pro.git ~/.fonts/adobe-fonts/source-code-pro
fc-cache -f -v ~/.fonts/adobe-fonts/source-code-pro
```

In addition, there are some Sublime options that is good to change from the start. Below you can see the changes I did to my User settings file (**Preferences** -> **Settings**).

```javascript
{
	"font_face": "Source Code Pro",
	"font_size": 10,
	"spell_check": true,
   	"always_show_minimap_viewport": true,
   	"highlight_line": true,

   	"trim_trailing_white_space_on_save": true,
   	"show_encoding": true,
   	"show_line_endings": true,

   	"dictionary": "Packages/Language - English/en_GB.dic",
}
```

## Conclusion

I am really impressed by the work Microsoft did with the **Windows Subsystem for Linux**. I am a Linux user for almost a decade but I always had those days that I had to access a Windows system, either for Microsoft Office, or a BIOS update, or something else. As a result, I always setup my machines to dual-boot Windows and Linux.

With WSL, I have a good feeling that it could be a nice alternative to dual boot. Apart from some limitations with GUI applications not running through an X-server, I didn't have any problem running my favourite languages and tools in WSL. Including Node.js, Racket, Go, Elixir and Erlang, and of course native Linux tools like SSH.

In conclusion, nicely done by Microsoft, and I really hope that this will continue to improve and get polished in order to reach the stability levels we require for day-to-day usage. This could be what Microsoft needed to make Windows a viable option for developers loving their Linux!

<h2 id="problem">Problem</h2>
While I was testing out the <a href="https://msdn.microsoft.com/en-gb/commandline/wsl/install_guide">Windows Subsystem for Linux - WSL</a> I setup all my favourite programming languages inside the Linux system instead of Windows. Everything was working smoothly until I had to use an editor to work on some projects.
I initially installed <a href="https://code.visualstudio.com/">Visual Studio Code</a> in my Windows system but realised that I don’t have access to the <code>PATH</code> or the binaries of my languages installed inside WSL. The solution is to have an X-server running in Windows and then run the editor from inside the WSL but rendering to the X-server display (easier than it sounds).
BTW, if you are a VIM user you can stop reading right now since VIM works out of the box.
<h2 id="solution">Solution</h2>
Firstly, we need to install an X-server in Windows. I installed <a href="https://sourceforge.net/projects/xming/">Xming X Server for Windows</a> and it works like a charm, with all the default options. Just download, install, and run!
If the server is running successfully, you should see an icon in the notification area of the taskbar that says Xming Server:0.0.
Once the X-server is running we are ready to go. In order to start an application with a GUI from inside WSL and have it render in Windows we need to set the environment variable <code>DISPLAY=:0</code> and then run the command.
<pre><code class="language-bash"># Either set it right before the command
DISPLAY=:0 command-with-a-gui

# or set it for the whole terminal session
export DISPLAY=:0
command-with-a-gui</code></pre>
<h3 id="visual-studio-code">Visual Studio Code</h3>
I started using Visual Studio Code for the most part of last year and I was really happy with it, but it seems that it cannot properly run through an X server and it is <a href="https://github.com/Microsoft/vscode/issues/13138">not a priority of its team to get it working</a>.
Not a huge issue since my previously favourite editor is perfectly working, but still a bummer since Microsoft’s own editor does not work with their system!
<h3 id="sublime-text-3">Sublime Text 3</h3>
The <a href="https://www.sublimetext.com/docs/3/linux_repositories.html">Sublime Text 3 installation instructions</a> are straightforward but I copy the commands needed here to make the article more complete.
<pre><code class="language-bash">wget -qO - https://download.sublimetext.com/sublimehq-pub.gpg | sudo apt-key add -
echo &quot;deb https://download.sublimetext.com/ apt/stable/&quot; | sudo tee /etc/apt/sources.list.d/sublime-text.list
sudo apt-get update &amp;&amp; sudo apt-get install sublime-text</code></pre>
The command to run Sublime Text 3 is <code>subl</code>. So in conjunction with the X-server display variable we can run Sublime using the following command:
<pre><code class="language-bash">DISPLAY=:0 subl</code></pre>
The above command should open a window of Sublime Text inside Windows, and if you try to Open folder you should be seeing the Linux subsystem file-system.
The next step is to <a href="https://packagecontrol.io/installation">install the package control</a> for Sublime so that we can install our favourite plugins.
<h2 id="next-steps">Next steps</h2>
The font I use for all my text editors, and IDEs, is <a href="https://github.com/adobe-fonts/source-code-pro/">Source Code Pro</a>, which in Ubuntu-like systems can be installed easily using the following commands:
<pre><code class="language-bash">git clone --depth 1 --branch release https://github.com/adobe-fonts/source-code-pro.git ~/.fonts/adobe-fonts/source-code-pro
fc-cache -f -v ~/.fonts/adobe-fonts/source-code-pro</code></pre>
In addition, there are some Sublime options that is good to change from the start. Below you can see the changes I did to my User settings file (Preferences -&gt; Settings).
<pre><code class="language-javascript">{
 &quot;font_face&quot;: &quot;Source Code Pro&quot;,
 &quot;font_size&quot;: 10,
 &quot;spell_check&quot;: true,
 &quot;always_show_minimap_viewport&quot;: true,
 &quot;highlight_line&quot;: true,

 &quot;trim_trailing_white_space_on_save&quot;: true,
 &quot;show_encoding&quot;: true,
 &quot;show_line_endings&quot;: true,

 &quot;dictionary&quot;: &quot;Packages/Language - English/en_GB.dic&quot;,
}</code></pre>
<h2 id="conclusion">Conclusion</h2>
I am really impressed by the work Microsoft did with the Windows Subsystem for Linux. I am a Linux user for almost a decade but I always had those days that I had to access a Windows system, either for Microsoft Office, or a BIOS update, or something else. As a result, I always setup my machines to dual-boot Windows and Linux.
With WSL, I have a good feeling that it could be a nice alternative to dual boot. Apart from some limitations with GUI applications not running through an X-server, I didn’t have any problem running my favourite languages and tools in WSL. Including Node.js, Racket, Go, Elixir and Erlang, and of course native Linux tools like SSH.
In conclusion, nicely done by Microsoft, and I really hope that this will continue to improve and get polished in order to reach the stability levels we require for day-to-day usage. This could be what Microsoft needed to make Windows a viable option for developers loving their Linux!

In this tutorial I describe how to setup Sublime Text 3 editor with Linux Subsystem in Windows 10 to run it from inside the Ubuntu system.

Windows Subsystem for Linux - Editor (GUI) Setup


## Problem

I recently started learning [Racket](https://racket-lang.org/) and one of the first things I do with a new language is integrating it with [AWS Lambda](https://aws.amazon.com/lambda/).

One of my favourite languages is [Go](https://golang.org/), and since both Racket and Go can be compiled down to self-contained executable binaries we can re-use some of the knowledge running Go on AWS Lambda.

## AWS Lambda Overview

Before delving into the detailed solution that works best for me I will provide the different ways we can run code in Lambda.

### Language natively supported by AWS Lambda

If your language is supported natively by Lambda then it's very easy and you should follow the AWS documentation for the language. As of the time of writing, the [supported languages](http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html) include **Node.js**, **Java**, **Python**, and **.NET Core**.

### Language can be compiled to C shared library

If your language of choice can be compiled down to a shared C-library binary, then the most performant way to run on Lambda, is to use a wrapper in Python that loads this library and directly make calls to your shared library.

This is the best way I have found to run Go code so far and I make use of the great library [eawsy/aws-lambda-go](https://github.com/eawsy/aws-lambda-go) (For Go 1.8+ there is a [new version](https://github.com/eawsy/aws-lambda-go-shim) which uses Go plugins).

This approach can be used to run any language compiled down to a C shared library.

### Language can be compiled to standalone binary that runs on Amazon Linux

If your language of choice cannot be compiled to shared library, but can be compiled to standalone binary that can run on the Amazon Linux system (which is the OS AWS Lambda uses behind the scenes) then it's very easy to run the application binary through Lambda.

The easiest and simplest way to run your code is to spawn a new subprocess of your binary using some lambda wrapper code (in Python or Node.js) and pass the ```event``` and/or ```context``` as input to that process. Then, you read the standard output of the subprocess from inside the wrapper code and return it to the caller of your Lambda code.

The only downside of this approach is that each Lambda function invocation is pretty slow, since a new process has to be started every time your function runs.

For example, using the code from [Daniel's Brunner blog post](http://www.dbrunner.net/2015/08/27/running-racket-on-aws-lambda/) each Lambda invocation configured with **128MB** memory has an average runtime of **~450-550ms**. You can boost the performance of this if you increase your Lambda's memory to the maximum of **1536MB**. This will bring the average runtime down to **~50-150ms**.

The reason behind this interesting fact is that increasing your Lambda function's memory, you also increase its CPU power, hence leading to much faster subprocess spawning times.

This spawning per function invocation works fine, but it's still slow for me and I don't want to pay for the maximum Lambda memory to get good performance. My proposed solution is a very simple adaptation of this approach.

Since Lambda has a static initialisation section every time the underlying container is started, I will spawn a process during that time, and then in each function invocation I will communicate over **standard input and output** with the subprocess from inside the lambda wrapper code.

This has a tremendous speedup over creating a new process every time since the average invocation runtime for the **128MB** using this approach is **1-100ms**, and consistently stays under **~10ms** with the **1536MB** memory configuration. This speedup is significant for lambdas that run many times over a period of time because the overhead of spawning a process is only observed the first time our code will run in a specific container instance, and then the same process is re-used leading to these extremely fast times.

See proof below, using the 128MB memory configuration, and runtime of **0.51ms** (yes that's less than a millisecond)!

![AWS Lambda running Racket](/articles/aws-lambda-meets-racket/aws-lambda-racket-128mb.png "AWS Lambda running Racket")

## Solution

As explained in the previous section, my best solution so far which keeps the complexity to a minimum, is to spawn a subprocess of our Racket application binary during the static initialisation of the Lambda function and re-use that process during the individual lambda invocations, by communicating over stdio.

### Racket application

I will use the following Racket code as example, which just echoes back the input of the application.

```scheme
#lang racket/base

;; This is the actual logic of our code!
(define (execute-logic data)
  (display (format "data: ~a~%" data))
  (flush-output))

;; The following code waits for one line of input and then dispatches it to the `execute-logic` function.
;; This way we can have full control over what we can do and there can be an arbitrarily complex protocol
;; between the caller and this application over **stdio**.
(define (loopInput)
  (execute-logic (read-line))
  (loopInput))
(loopInput)
```

The last four lines in the code above is just a loop that reads a line from standard input, and then calls our ```execute-logic``` function passing the data received. This is the only boilerplate needed by our application code.

Your logic can do whatever it wants with the input data and then just print the result to standard output. Here we just use the [display](https://docs.racket-lang.org/reference/Writing.html#%28def._%28%28quote._~23~25kernel%29._display%29%29) function to print the input to standard output.

### Wrapper code in Node.js

**Update@2017-10-18:** I created an npm package to significantly reduce the boilerplate code shown below, so **after** you have read this article and understood how the solution works, use the package [AWS-lambda-binary](https://www.npmjs.com/package/aws-lambda-binary) in your production Lambda functions.

The wrapper code will be a bit longer but still remains very simple to understand.

```javascript
const child_process = require('child_process');
const readline = require('readline');

/****************************
 * START OF WRAPPER CODE
 ****************************/
const execPath = './application';

let proc = initProc();

function initProc(options) {
    options = options || {};
    const p = child_process.spawn(execPath);

    // Add your own custom handler if you want to handle the errors differently.
    p.stderr.on('data', (err) => {
        console.error('proc stderr: ', err);
    });

    // You might want to use ```exit``` event instead of ```close``` if you don't
    // care about the ```stdio streams``` of the subprocess.
    // https://nodejs.org/api/child_process.html#child_process_event_close
    // https://nodejs.org/api/child_process.html#child_process_event_exit
    p.on('close', function (code) {
        if (code !== 0) {
            console.error(new Error(`Process closed with code: ${code}`));
        }
        const {handlerProcCloseCallback} = proc;
        proc = null;
        if (handlerProcCloseCallback) {
            handlerProcCloseCallback(code);
        }
    });

    // This is the part that you get the result back from your Racket application
    // I prefer to receive lines back from the application for simplicity
    // so I use https://nodejs.org/api/readline.html#readline_event_line
    // but you can adapt this to use binary data as well, exactly like we did with
    // `stderr` above.
    const rl = readline.createInterface({ input: p.stdout });
    rl.on('line', (line) => {
        const {handlerCallback} = proc;
        if (handlerCallback) {
            handlerCallback(line);
        }
    });

    return {
        p, rl,

        // Will be called for every **line** output from our application.
        handlerCallback: null,

        // Will be called when the application process is closed. You can use
        // this callback to restart it automatically or do something custom.
        handlerProcCloseCallback: null,
    };
}

function ensureProcRuns(options) {
    if (!proc) {
        proc = initProc(options);
    }
    if (options.resetCallbacks) {
        proc.handlerCallback = null;
        proc.handlerProcCloseCallback = null;
    }
}
/**************************
 * END OF WRAPPER CODE
 **************************/

exports.handler = function (event, context) {
    ensureProcRuns({resetCallbacks: true});

    // Register the handler we want for each line response!
    proc.handlerCallback = (result) => {
        console.log(`rkt: ${result}`);
        context.done(null, `result: ${result}`);
    };

    // Send the input to the Racket application
    proc.p.stdin.write(`${JSON.stringify({event, context})}\n`);
};

```

I don't think it needs lots of explanation but I will explain some of the important bits.

Your main focus should be inside the ```exports.handler = function(...) {...}``` section, and everything before that is just the wrapper code around the subprocess spawning.

The first thing we need to do inside our handler code is to call ```ensureProcRuns()``` in order to make sure that there is a running subprocess of our Racket application. Ideally, this should always be instant since we already instantiate a process during the initialisation phase of the container. If the subprocess fails though and exits during an invocation, this will ensure that future invocations will re-spawn the process.

After that first line, we need to register a callback function that will handle the result back from the Racket application. In my case here we just read a single line and then finish the lambda invocation with success ```context.done(...)``` and with return value whatever the line we received was.

It is very important to understand that this solution is not limited to single line responses. You can have your own protocol that spans multiple lines of response from the application, or even going down to binary data instead of line-by-line.

Finally, once we set our handler callback, we write on standard input of the subprocess the input to our Racket application. Here I just pass a JSON serialised object containing both the **event** and the **context** of the lambda invocation, which should cover most of your use-cases. If you want to avoid JSON deserialisation in your Racket code, then you can parse the event in this wrapper code and just pass a line of comma-separated values down to the application. Again, here I use a single line of input, but you can easily adapt this to span multiple lines of input or binary data.

There is also an additional callback that you can set, the ```handlerProcCloseCallback()```. This callback is called every time the subprocess's standard input and output streams are closed (usually when the subprocess terminates). For example, you can use this callback to re-spawn the closed subprocess and speedup future invocations, as shown below.

```javascript
proc.handlerProcCloseCallback = (code) => {
    ensureProcRuns({resetCallbacks: true});
};
```

## Compile and bundle your code

* Copy the Racket code from above and save it in a file named ```application.rkt```
* Compile the Racket code into a standalone binary

    ```bash
    raco exe --orig-exe application.rkt
    ```

* Copy the Node.js code from above and save it in a file named ```wrapper.js```
* Bundle everything together ```zip bundle.zip application wrapper.js```

You can upload ```bundle.zip``` now to your lambda function and test it.

Make sure that your lambda function has the following configuration:

| Configuration Property | Value |
|----------|-------|
|**Runtime**|Node.js 6.10|
|**Handler**|wrapper.handler|

Test around with different memory configuration to find a good balance between latency and cost for your application.

## Conclusion

As you can see, we did not invent quantum computing. We just applied old-fashioned programming principles to AWS Lambda. We have our custom application logic written in Racket, or any other compiled language, that accepts data from standard input, and writes to standard output. Anybody used Linux command line tools before? Yes, it is the same!

This solution is very flexible and allows you to either go with a minimalistic protocol of single line input and single line output as I have done above, or even with a super complicated custom protocol. After all, it's just standard input and standard output.

The most fascinating thing about this approach is that now your custom code can run in AWS Lambda with super low latencies!

SEO tags: #AWS #Racketlang - #AWSLambda #Meets #Racket


<h2 id="problem">Problem</h2>
I recently started learning <a href="https://racket-lang.org/">Racket</a> and one of the first things I do with a new language is integrating it with <a href="https://aws.amazon.com/lambda/">AWS Lambda</a>.
One of my favourite languages is <a href="https://golang.org/">Go</a>, and since both Racket and Go can be compiled down to self-contained executable binaries we can re-use some of the knowledge running Go on AWS Lambda.
<h2 id="aws-lambda-overview">AWS Lambda Overview</h2>
Before delving into the detailed solution that works best for me I will provide the different ways we can run code in Lambda.
<h3 id="language-natively-supported-by-aws-lambda">Language natively supported by AWS Lambda</h3>
If your language is supported natively by Lambda then it’s very easy and you should follow the AWS documentation for the language. As of the time of writing, the <a href="http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html">supported languages</a> include Node.js, Java, Python, and .NET Core.
<h3 id="language-can-be-compiled-to-c-shared-library">Language can be compiled to C shared library</h3>
If your language of choice can be compiled down to a shared C-library binary, then the most performant way to run on Lambda, is to use a wrapper in Python that loads this library and directly make calls to your shared library.
This is the best way I have found to run Go code so far and I make use of the great library <a href="https://github.com/eawsy/aws-lambda-go">eawsy/aws-lambda-go</a> (For Go 1.8+ there is a <a href="https://github.com/eawsy/aws-lambda-go-shim">new version</a> which uses Go plugins).
This approach can be used to run any language compiled down to a C shared library.
<h3 id="language-can-be-compiled-to-standalone-binary-that-runs-on-amazon-linux">Language can be compiled to standalone binary that runs on Amazon Linux</h3>
If your language of choice cannot be compiled to shared library, but can be compiled to standalone binary that can run on the Amazon Linux system (which is the OS AWS Lambda uses behind the scenes) then it’s very easy to run the application binary through Lambda.
The easiest and simplest way to run your code is to spawn a new subprocess of your binary using some lambda wrapper code (in Python or Node.js) and pass the <code>event</code> and/or <code>context</code> as input to that process. Then, you read the standard output of the subprocess from inside the wrapper code and return it to the caller of your Lambda code.
The only downside of this approach is that each Lambda function invocation is pretty slow, since a new process has to be started every time your function runs.
For example, using the code from <a href="http://www.dbrunner.net/2015/08/27/running-racket-on-aws-lambda/">Daniel’s Brunner blog post</a> each Lambda invocation configured with 128MB memory has an average runtime of ~450-550ms. You can boost the performance of this if you increase your Lambda’s memory to the maximum of 1536MB. This will bring the average runtime down to ~50-150ms.
The reason behind this interesting fact is that increasing your Lambda function’s memory, you also increase its CPU power, hence leading to much faster subprocess spawning times.
This spawning per function invocation works fine, but it’s still slow for me and I don’t want to pay for the maximum Lambda memory to get good performance. My proposed solution is a very simple adaptation of this approach.
Since Lambda has a static initialisation section every time the underlying container is started, I will spawn a process during that time, and then in each function invocation I will communicate over standard input and output with the subprocess from inside the lambda wrapper code.
This has a tremendous speedup over creating a new process every time since the average invocation runtime for the 128MB using this approach is 1-100ms, and consistently stays under ~10ms with the 1536MB memory configuration. This speedup is significant for lambdas that run many times over a period of time because the overhead of spawning a process is only observed the first time our code will run in a specific container instance, and then the same process is re-used leading to these extremely fast times.
See proof below, using the 128MB memory configuration, and runtime of 0.51ms (yes that’s less than a millisecond)!
<img src="/articles/aws-lambda-meets-racket/aws-lambda-racket-128mb.png" alt="AWS Lambda running Racket" title="AWS Lambda running Racket"/>
<h2 id="solution">Solution</h2>
As explained in the previous section, my best solution so far which keeps the complexity to a minimum, is to spawn a subprocess of our Racket application binary during the static initialisation of the Lambda function and re-use that process during the individual lambda invocations, by communicating over stdio.
<h3 id="racket-application">Racket application</h3>
I will use the following Racket code as example, which just echoes back the input of the application.
<pre><code class="language-scheme">#lang racket/base

;; This is the actual logic of our code!
(define (execute-logic data)
 (display (format &quot;data: ~a~%&quot; data))
 (flush-output))

;; The following code waits for one line of input and then dispatches it to the `execute-logic` function.
;; This way we can have full control over what we can do and there can be an arbitrarily complex protocol
;; between the caller and this application over **stdio**.
(define (loopInput)
 (execute-logic (read-line))
 (loopInput))
(loopInput)</code></pre>
The last four lines in the code above is just a loop that reads a line from standard input, and then calls our <code>execute-logic</code> function passing the data received. This is the only boilerplate needed by our application code.
Your logic can do whatever it wants with the input data and then just print the result to standard output. Here we just use the <a href="https://docs.racket-lang.org/reference/Writing.html#%28def._%28%28quote._~23~25kernel%29._display%29%29">display</a> function to print the input to standard output.
<h3 id="wrapper-code-in-nodejs">Wrapper code in Node.js</h3>
Update@2017-10-18: I created an npm package to significantly reduce the boilerplate code shown below, so after you have read this article and understood how the solution works, use the package <a href="https://www.npmjs.com/package/aws-lambda-binary">AWS-lambda-binary</a> in your production Lambda functions.
The wrapper code will be a bit longer but still remains very simple to understand.
<pre><code class="language-javascript">const child_process = require(&#39;child_process&#39;);
const readline = require(&#39;readline&#39;);

/****************************
 * START OF WRAPPER CODE
 ****************************/
const execPath = &#39;./application&#39;;

let proc = initProc();

function initProc(options) {
 options = options || {};
 const p = child_process.spawn(execPath);

 // Add your own custom handler if you want to handle the errors differently.
 p.stderr.on(&#39;data&#39;, (err) =&gt; {
 console.error(&#39;proc stderr: &#39;, err);
 });

 // You might want to use ```exit``` event instead of ```close``` if you don&#39;t
 // care about the ```stdio streams``` of the subprocess.
 // https://nodejs.org/api/child_process.html#child_process_event_close
 // https://nodejs.org/api/child_process.html#child_process_event_exit
 p.on(&#39;close&#39;, function (code) {
 if (code !== 0) {
 console.error(new Error(`Process closed with code: ${code}`));
 }
 const {handlerProcCloseCallback} = proc;
 proc = null;
 if (handlerProcCloseCallback) {
 handlerProcCloseCallback(code);
 }
 });

 // This is the part that you get the result back from your Racket application
 // I prefer to receive lines back from the application for simplicity
 // so I use https://nodejs.org/api/readline.html#readline_event_line
 // but you can adapt this to use binary data as well, exactly like we did with
 // `stderr` above.
 const rl = readline.createInterface({ input: p.stdout });
 rl.on(&#39;line&#39;, (line) =&gt; {
 const {handlerCallback} = proc;
 if (handlerCallback) {
 handlerCallback(line);
 }
 });

 return {
 p, rl,

 // Will be called for every **line** output from our application.
 handlerCallback: null,

 // Will be called when the application process is closed. You can use
 // this callback to restart it automatically or do something custom.
 handlerProcCloseCallback: null,
 };
}

function ensureProcRuns(options) {
 if (!proc) {
 proc = initProc(options);
 }
 if (options.resetCallbacks) {
 proc.handlerCallback = null;
 proc.handlerProcCloseCallback = null;
 }
}
/**************************
 * END OF WRAPPER CODE
 **************************/

exports.handler = function (event, context) {
 ensureProcRuns({resetCallbacks: true});

 // Register the handler we want for each line response!
 proc.handlerCallback = (result) =&gt; {
 console.log(`rkt: ${result}`);
 context.done(null, `result: ${result}`);
 };

 // Send the input to the Racket application
 proc.p.stdin.write(`${JSON.stringify({event, context})}\n`);
};
</code></pre>
I don’t think it needs lots of explanation but I will explain some of the important bits.
Your main focus should be inside the <code>exports.handler = function(...) {...}</code> section, and everything before that is just the wrapper code around the subprocess spawning.
The first thing we need to do inside our handler code is to call <code>ensureProcRuns()</code> in order to make sure that there is a running subprocess of our Racket application. Ideally, this should always be instant since we already instantiate a process during the initialisation phase of the container. If the subprocess fails though and exits during an invocation, this will ensure that future invocations will re-spawn the process.
After that first line, we need to register a callback function that will handle the result back from the Racket application. In my case here we just read a single line and then finish the lambda invocation with success <code>context.done(...)</code> and with return value whatever the line we received was.
It is very important to understand that this solution is not limited to single line responses. You can have your own protocol that spans multiple lines of response from the application, or even going down to binary data instead of line-by-line.
Finally, once we set our handler callback, we write on standard input of the subprocess the input to our Racket application. Here I just pass a JSON serialised object containing both the event and the context of the lambda invocation, which should cover most of your use-cases. If you want to avoid JSON deserialisation in your Racket code, then you can parse the event in this wrapper code and just pass a line of comma-separated values down to the application. Again, here I use a single line of input, but you can easily adapt this to span multiple lines of input or binary data.
There is also an additional callback that you can set, the <code>handlerProcCloseCallback()</code>. This callback is called every time the subprocess’s standard input and output streams are closed (usually when the subprocess terminates). For example, you can use this callback to re-spawn the closed subprocess and speedup future invocations, as shown below.
<pre><code class="language-javascript">proc.handlerProcCloseCallback = (code) =&gt; {
 ensureProcRuns({resetCallbacks: true});
};</code></pre>
<h2 id="compile-and-bundle-your-code">Compile and bundle your code</h2>
<ul>
<li>Copy the Racket code from above and save it in a file named <code>application.rkt</code>
</li>
<li>Compile the Racket code into a standalone binary
<pre><code class="language-bash"> raco exe --orig-exe application.rkt</code></pre>
</li>
<li>Copy the Node.js code from above and save it in a file named <code>wrapper.js</code>
</li>
<li>Bundle everything together <code>zip bundle.zip application wrapper.js</code>
</li>
</ul>
You can upload <code>bundle.zip</code> now to your lambda function and test it.
Make sure that your lambda function has the following configuration:
<table>
<thead>
<tr>
<th>Configuration Property</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>Runtime</td>
<td>Node.js 6.10</td>
</tr>
<tr>
<td>Handler</td>
<td>wrapper.handler</td>
</tr>
</tbody></table>
Test around with different memory configuration to find a good balance between latency and cost for your application.
<h2 id="conclusion">Conclusion</h2>
As you can see, we did not invent quantum computing. We just applied old-fashioned programming principles to AWS Lambda. We have our custom application logic written in Racket, or any other compiled language, that accepts data from standard input, and writes to standard output. Anybody used Linux command line tools before? Yes, it is the same!
This solution is very flexible and allows you to either go with a minimalistic protocol of single line input and single line output as I have done above, or even with a super complicated custom protocol. After all, it’s just standard input and standard output.
The most fascinating thing about this approach is that now your custom code can run in AWS Lambda with super low latencies!
SEO tags: #AWS #Racketlang - #AWSLambda #Meets #Racket

In this tutorial I provide an easy way to run any compiled language with AWS Lambda with communication over standard input and output.

AWS Lambda meets Racket (or any compiled language)

## Problem

Few months ago I started receiving the following warning from [Github](https://github.com), **Your GitHub academic discount coupon has expired**, and if I wanted to continue working with my private repositories I would have to pay.
In addition to this, for the last couple of months I was investigating and trying different products for personal and **private** note taking. I found a lot of interesting products that were integrating with git services like Github, or [Gitlab](https://gitlab.com) and I was really close to settle with them, but I went deeper into their paid offerings and they actually did not provide encryption at rest, so it was a **no-go**.

Of course, someone can say they are used by millions, so why not trust them. Well, I don't, and in conjunction with their availabilities issues I decided to find alternatives. Actually, while I was writing this article Gitlab was **out of service** with the website returning **500 internal server** errors.

My next solution was a custom website I would develop that would have a super-simple UI, including an editor and a list of all my notes, which would use either [Google Drive](https://www.google.com/drive/) or [Amazon S3](https://aws.amazon.com/s3/) as it's backend, and since I would code the whole thing I would add client encryption to my super important notes (like passwords).

I started looking around for websites similar to this, and found plenty, but none provided the privacy I wanted since they would pass my data through their servers, and although they would say they don't look at them, well I was skeptical! I found some very good products though which I would gladly use if it wasn't for the privacy issues, like [Write-box](http://writeboxapps.com/), [Standard Notes](https://app.standardnotes.org/) and [Dynalist](https://dynalist.io/).

So long story short, nothing suited my needs which are summarised below:

* **Privacy** of my data with encryption at rest
* **Accessibility** and **Availability** of my notes from my laptops and my smartphone

I was close to start implementing my solution, but then I looked closely into every coder's friend, [Git](https://git-scm.com/).

## Solution

I decided to use git repositories for storing my notes, and files. After investigating several self-hosted solutions, including gitlab's, and bitbucket's server offerings, I settled on an open source super lightweight self-hosted git server, [Gitea](https://gitea.io/). Gitea is written in [Go](https://golang.org/) which allows for amazingly low resource consumption (it uses less than 0.3% CPU on my T2.nano EC2 instance).

In addition to the self-hosted git server, I found out about the **git bare repos** which basically act like a git server when hosted on a system accessible over SSH. I use these bare repos for super simple stuff where I don't need the accessibility over my phone, like coding side projects.

### Git bare repositories

Due to the design of git, you can have a directory acting as the **remote location** of all your working directories, where you can push and pull your files.

How to setup a git repository which can be used as remote location?

1. On the server (EC2 instance, or any other system accessible through SSH) run the following command to create the git directory:

 ```bash
 mkdir -p $HOME/git-repos/<REPO_NAME>.git && cd $HOME/git-repos/<REPO_NAME>.git
 git init --bare
 ```

 The first command creates an empty directory where our files will be and the second one initialises that directory to hold a git repository. The ```--bare``` argument is the important bit, which tells git to create a repository without a **working directory**, meaning that nobody will use this directory as his workspace to work on the files. It will be simply used as a remote location for other clones of this git repository.

2. On any system that you want to work with the above repository just clone it and work with it as you would with any other git repo.

 ```bash
 git clone username@serverHostname:~/git-repos/<REPO-NAME>.git
 ```

The good thing of git bare repositories is that they can be used from any system that has access to the server hosting them. I know that lots of people are using this approach but instead of hosting their **bare repos** on a server, they store them in Dropbox or Google Drive. They achieve the accessibility and availability aspect as long as they keep their systems in-sync, but the privacy aspect still resides in the amount of trust you have on Dropbox and Google. BTW, I am a huge fan of Google services like Google Drive and Google Keep, which I use for pretty much all my storage and note taking needs. However, there are some stuff for which I need some extra privacy (i.e. nuclear codes)!

### Self hosted git server

As I said, my git server of choice, at least for the time being, is Gitea, especially due to the low resource consumption on my server. It provides a nice Github-like web interface which you can use to read and edit your files using any web browser, and also has simple organisation of the repositories it creates just like any normal git bare repo which you can use like explained above if you want to bypass the server.

It is amazingly simple to setup as described in https://docs.gitea.io/en-us/install-from-binary/.

```bash
# Download the git server
wget -O gitea https://dl.gitea.io/gitea/1.0.1/gitea-1.0.1-linux-amd64 && chmod u+x gitea

# Run the server
./gitea web
# Or Run the server in the background
nohup ./gitea web &
```

Once you have the server running, access it through a web-browser and you will be prompted for the first setup, which will ask you to register a user (automatically becomes the admin user as well), and to choose your database (SQLite3 should be sufficient without other dependencies).

Obviously, you need to have access to your server over HTTP or HTTPS preferably.

As a side note, I have to mention that Gitea stores all the git repositories you create through its web interface in a directory you define during the first setup and these repositories are just like any git bare repository you create on your own. In a nutshell, Gitea is just a web proxy to git bare repos that does user management and provides a web interface to them.

### Backups and encryption

OK, I achieved availability and accessibility since I can access my git server from anywhere using a web browser or over an SSH connection. We achieved incredibly easy management of my files through any editor and any system that can manage git repositories, without forcing us to use a single website UI. But what happens with encryption and privacy?

In my case, I use a T2.nano EC2 instance as my server which I use for personal projects few years now. The [EBS volume](https://aws.amazon.com/ebs/) backing the instance is using [Amazon EBS Encryption](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html) so we achieve our encryption needs. The security of the server itself is another topic but following AWS best-practices around IAM security rules it should be fine.

In addition, just to avoid any problems when that horribly dark day comes when my host will die or stop working, I take backups of all my repos and store them in [S3 with server-side encryption](http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html) after every ```push``` I do on the repos. I also take weekly backups of the whole EBS which will make it very easy to just spin up another server with the exact configuration if this one fails.

The script I use for my repository backups is as shown below, and it assumes that all the git repositories I create, on my own or through Gitea, are under the ```$HOME/git-repos``` path.

```bash
#!/bin/bash

set -e

GIT_REPO_DIR="git-repos"
S3_BUCKET="s3://your-s3-bucket-name/git-repos/"

TIMENOW="$(date --rfc-3339=seconds | tr ' ' '_' | tr ':' '-' | tr '+' '.')"
FILENAME="git-repos.$TIMENOW.tgz"

DEST_TAR="/tmp/$FILENAME"

tar -cvzf "$DEST_TAR" -C "$HOME" "$GIT_REPO_DIR"

aws s3 cp "$DEST_TAR" "$S3_BUCKET$FILENAME" --storage-class STANDARD_IA --sse
```

If you don't want to get into the trouble of enabling webhooks in Gitea to backup after every push, you can just do periodic backups using cronjobs. Assuming that the above backup script file is located in ```/home/user/bin/backup-git-repos-s3.sh``` a simple cronjob to take a backup every day at 20:00 is shown below.

```bash
crontab -e

0 20 * * * /home/user/bin/backup-git-repos-s3.sh > /tmp/crontab.backup-git-repos.log 2>&1
```

I personally use Amazon S3 for my backups cause I already use it for hosting all my static websites and for general backup storage since it's dirty cheap. You could easily modify the script above to upload the file in Google Drive, or any other preferred service.

## Conclusion

The first initial setup for the Git self hosted server and the backup automation might seem more work than just signing up on an online service, but for me every it totally paid off, especially since I already had a server running. I have a super flexible way to keep track of personal projects I don't want to have public, or to store sensitive information without being afraid of prying employee eyes. And with the backups, in case something goes wrong with my server I have perfectly good archives to work with.

Although I solved my problem, I still wish there was a **completely client-side web application** integrated with Google Drive or Amazon S3, which will provide capability to encrypt my notes before storing them, without transmission of any data to third-parties (apart of course from the storage service itself).

To be honest, I was super amazed that there are so many online products integrating with Github, Dropbox, Google Drive, and others, but still none of them provides privacy and client-side encryption before sending the data over. Therefore, I still plan to implement that super-duper web application, so keep an eye for it over the next coming months :)

<h2 id="problem">Problem</h2>
Few months ago I started receiving the following warning from <a href="https://github.com">Github</a>, Your GitHub academic discount coupon has expired, and if I wanted to continue working with my private repositories I would have to pay.
In addition to this, for the last couple of months I was investigating and trying different products for personal and private note taking. I found a lot of interesting products that were integrating with git services like Github, or <a href="https://gitlab.com">Gitlab</a> and I was really close to settle with them, but I went deeper into their paid offerings and they actually did not provide encryption at rest, so it was a no-go.
Of course, someone can say they are used by millions, so why not trust them. Well, I don’t, and in conjunction with their availabilities issues I decided to find alternatives. Actually, while I was writing this article Gitlab was out of service with the website returning 500 internal server errors.
My next solution was a custom website I would develop that would have a super-simple UI, including an editor and a list of all my notes, which would use either <a href="https://www.google.com/drive/">Google Drive</a> or <a href="https://aws.amazon.com/s3/">Amazon S3</a> as it’s backend, and since I would code the whole thing I would add client encryption to my super important notes (like passwords).
I started looking around for websites similar to this, and found plenty, but none provided the privacy I wanted since they would pass my data through their servers, and although they would say they don’t look at them, well I was skeptical! I found some very good products though which I would gladly use if it wasn’t for the privacy issues, like <a href="http://writeboxapps.com/">Write-box</a>, <a href="https://app.standardnotes.org/">Standard Notes</a> and <a href="https://dynalist.io/">Dynalist</a>.
So long story short, nothing suited my needs which are summarised below:
<ul>
<li>Privacy of my data with encryption at rest</li>
<li>Accessibility and Availability of my notes from my laptops and my smartphone</li>
</ul>
I was close to start implementing my solution, but then I looked closely into every coder’s friend, <a href="https://git-scm.com/">Git</a>.
<h2 id="solution">Solution</h2>
I decided to use git repositories for storing my notes, and files. After investigating several self-hosted solutions, including gitlab’s, and bitbucket’s server offerings, I settled on an open source super lightweight self-hosted git server, <a href="https://gitea.io/">Gitea</a>. Gitea is written in <a href="https://golang.org/">Go</a> which allows for amazingly low resource consumption (it uses less than 0.3% CPU on my T2.nano EC2 instance).
In addition to the self-hosted git server, I found out about the git bare repos which basically act like a git server when hosted on a system accessible over SSH. I use these bare repos for super simple stuff where I don’t need the accessibility over my phone, like coding side projects.
<h3 id="git-bare-repositories">Git bare repositories</h3>
Due to the design of git, you can have a directory acting as the remote location of all your working directories, where you can push and pull your files.
How to setup a git repository which can be used as remote location?
<ol>
<li>On the server (EC2 instance, or any other system accessible through SSH) run the following command to create the git directory:
<pre><code class="language-bash"> mkdir -p $HOME/git-repos/&lt;REPO_NAME&gt;.git &amp;&amp; cd $HOME/git-repos/&lt;REPO_NAME&gt;.git
 git init --bare</code></pre>
 The first command creates an empty directory where our files will be and the second one initialises that directory to hold a git repository. The <code>--bare</code> argument is the important bit, which tells git to create a repository without a working directory, meaning that nobody will use this directory as his workspace to work on the files. It will be simply used as a remote location for other clones of this git repository.
</li>
<li>On any system that you want to work with the above repository just clone it and work with it as you would with any other git repo.
<pre><code class="language-bash"> git clone username@serverHostname:~/git-repos/&lt;REPO-NAME&gt;.git</code></pre>
</li>
</ol>
The good thing of git bare repositories is that they can be used from any system that has access to the server hosting them. I know that lots of people are using this approach but instead of hosting their bare repos on a server, they store them in Dropbox or Google Drive. They achieve the accessibility and availability aspect as long as they keep their systems in-sync, but the privacy aspect still resides in the amount of trust you have on Dropbox and Google. BTW, I am a huge fan of Google services like Google Drive and Google Keep, which I use for pretty much all my storage and note taking needs. However, there are some stuff for which I need some extra privacy (i.e. nuclear codes)!
<h3 id="self-hosted-git-server">Self hosted git server</h3>
As I said, my git server of choice, at least for the time being, is Gitea, especially due to the low resource consumption on my server. It provides a nice Github-like web interface which you can use to read and edit your files using any web browser, and also has simple organisation of the repositories it creates just like any normal git bare repo which you can use like explained above if you want to bypass the server.
It is amazingly simple to setup as described in <a href="https://docs.gitea.io/en-us/install-from-binary/">https://docs.gitea.io/en-us/install-from-binary/</a>.
<pre><code class="language-bash"># Download the git server
wget -O gitea https://dl.gitea.io/gitea/1.0.1/gitea-1.0.1-linux-amd64 &amp;&amp; chmod u+x gitea

# Run the server
./gitea web
# Or Run the server in the background
nohup ./gitea web &amp;</code></pre>
Once you have the server running, access it through a web-browser and you will be prompted for the first setup, which will ask you to register a user (automatically becomes the admin user as well), and to choose your database (SQLite3 should be sufficient without other dependencies).
Obviously, you need to have access to your server over HTTP or HTTPS preferably.
As a side note, I have to mention that Gitea stores all the git repositories you create through its web interface in a directory you define during the first setup and these repositories are just like any git bare repository you create on your own. In a nutshell, Gitea is just a web proxy to git bare repos that does user management and provides a web interface to them.
<h3 id="backups-and-encryption">Backups and encryption</h3>
OK, I achieved availability and accessibility since I can access my git server from anywhere using a web browser or over an SSH connection. We achieved incredibly easy management of my files through any editor and any system that can manage git repositories, without forcing us to use a single website UI. But what happens with encryption and privacy?
In my case, I use a T2.nano EC2 instance as my server which I use for personal projects few years now. The <a href="https://aws.amazon.com/ebs/">EBS volume</a> backing the instance is using <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html">Amazon EBS Encryption</a> so we achieve our encryption needs. The security of the server itself is another topic but following AWS best-practices around IAM security rules it should be fine.
In addition, just to avoid any problems when that horribly dark day comes when my host will die or stop working, I take backups of all my repos and store them in <a href="http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html">S3 with server-side encryption</a> after every <code>push</code> I do on the repos. I also take weekly backups of the whole EBS which will make it very easy to just spin up another server with the exact configuration if this one fails.
The script I use for my repository backups is as shown below, and it assumes that all the git repositories I create, on my own or through Gitea, are under the <code>$HOME/git-repos</code> path.
<pre><code class="language-bash">#!/bin/bash

set -e

GIT_REPO_DIR=&quot;git-repos&quot;
S3_BUCKET=&quot;s3://your-s3-bucket-name/git-repos/&quot;

TIMENOW=&quot;$(date --rfc-3339=seconds | tr &#39; &#39; &#39;_&#39; | tr &#39;:&#39; &#39;-&#39; | tr &#39;+&#39; &#39;.&#39;)&quot;
FILENAME=&quot;git-repos.$TIMENOW.tgz&quot;

DEST_TAR=&quot;/tmp/$FILENAME&quot;

tar -cvzf &quot;$DEST_TAR&quot; -C &quot;$HOME&quot; &quot;$GIT_REPO_DIR&quot;

aws s3 cp &quot;$DEST_TAR&quot; &quot;$S3_BUCKET$FILENAME&quot; --storage-class STANDARD_IA --sse</code></pre>
If you don’t want to get into the trouble of enabling webhooks in Gitea to backup after every push, you can just do periodic backups using cronjobs. Assuming that the above backup script file is located in <code>/home/user/bin/backup-git-repos-s3.sh</code> a simple cronjob to take a backup every day at 20:00 is shown below.
<pre><code class="language-bash">crontab -e

0 20 * * * /home/user/bin/backup-git-repos-s3.sh &gt; /tmp/crontab.backup-git-repos.log 2&gt;&amp;1</code></pre>
I personally use Amazon S3 for my backups cause I already use it for hosting all my static websites and for general backup storage since it’s dirty cheap. You could easily modify the script above to upload the file in Google Drive, or any other preferred service.
<h2 id="conclusion">Conclusion</h2>
The first initial setup for the Git self hosted server and the backup automation might seem more work than just signing up on an online service, but for me every it totally paid off, especially since I already had a server running. I have a super flexible way to keep track of personal projects I don’t want to have public, or to store sensitive information without being afraid of prying employee eyes. And with the backups, in case something goes wrong with my server I have perfectly good archives to work with.
Although I solved my problem, I still wish there was a completely client-side web application integrated with Google Drive or Amazon S3, which will provide capability to encrypt my notes before storing them, without transmission of any data to third-parties (apart of course from the storage service itself).
To be honest, I was super amazed that there are so many online products integrating with Github, Dropbox, Google Drive, and others, but still none of them provides privacy and client-side encryption before sending the data over. Therefore, I still plan to implement that super-duper web application, so keep an eye for it over the next coming months :)

I wanted a solution to store private information and projects I didn't want others to see. Self-hosted git repositories are my solution to that.

Self-hosted git repository for privacy and control

## Problem

While I was learning Elixir I wanted something to use the [for-comprehension](http://elixir-lang.org/getting-started/comprehensions.html) to solve a problem. I remembered the [Eight queens puzzle](https://en.wikipedia.org/wiki/Eight_queens_puzzle) back from my college courses so I decided to give it a go.

I adapted the problem a bit, so my solution takes as inputs the following:

```
Inputs
======
n: number of queens to position on the board
m: size of the board side

Outputs
=======
List of all the solutions (List[List[int]])
```

## Naive Solution

The solution is simple, and uses **backtracking** to iterate over all possibilities and select the valid ones.

```elixir
defmodule Queens do

 @doc """
 Given n number of queens and m the size of the checkerboard, find all solutions to 
 position each queen so that it does not collide with any other queen vertically, 
 horizontally or diagonally.
 """
 def solve(0, _m), do: [[]]
 def solve(n, m) do
 for done_queens <- solve(n-1, m),
 avail_pos <- (Enum.to_list(1..m) -- done_queens),
 safe_pos(avail_pos, done_queens, 1), 
 do: [avail_pos | done_queens]
 end

 defp safe_pos(_, [], _), do: true
 defp safe_pos(pos, [queen | queens], distance) do
 (pos != queen + distance) and 
 (pos != queen - distance) and 
 safe_pos(pos, queens, distance+1)
 end

end
```

Obviously, this is **not** the fastest algorithm to solve this problem (exponential complexity), but it shows how elegant the solution can be using Elixir's comprehensions.

Sample output in **iex**:

```elixir
iex(47)> c("queens.ex")
[Queens]
iex(48)> :io.write Queens.solve(4, 4)
[[3,1,4,2],[2,4,1,3]]:ok
iex(49)> :io.write Queens.solve(3, 4)
[[2,4,1],[1,4,2],[4,1,3],[3,1,4]]:ok
iex(50)> :io.write Queens.solve(5, 5)
[[4,2,5,3,1],[3,5,2,4,1],[5,3,1,4,2],[4,1,3,5,2],[5,2,4,1,3],[1,4,2,5,3],[2,5,3,1,4],[1,3,5,2,4],[3,1,4,2,5],[2,4,1,3,5]]:ok
iex(51)> :io.write Queens.solve(1, 5)
[[1],[2],[3],[4],[5]]:ok
```

**Demo:** http://elixirplayground.com?gist=688be58a64712a172878a58683ed0eda

## Conclusion

Every day I learn more and more Elixir (and Erlang) and I can say that it's among the few languages that managed to keep me interested and excited for more than 3-4 months (looking at you Scala and Python).

<h2 id="problem">Problem</h2>
While I was learning Elixir I wanted something to use the <a href="http://elixir-lang.org/getting-started/comprehensions.html">for-comprehension</a> to solve a problem. I remembered the <a href="https://en.wikipedia.org/wiki/Eight_queens_puzzle">Eight queens puzzle</a> back from my college courses so I decided to give it a go.
I adapted the problem a bit, so my solution takes as inputs the following:
<pre><code>Inputs
======
n: number of queens to position on the board
m: size of the board side

Outputs
=======
List of all the solutions (List[List[int]])</code></pre><h2 id="naive-solution">Naive Solution</h2>
The solution is simple, and uses backtracking to iterate over all possibilities and select the valid ones.
<pre><code class="language-elixir">defmodule Queens do

 @doc &quot;&quot;&quot;
 Given n number of queens and m the size of the checkerboard, find all solutions to 
 position each queen so that it does not collide with any other queen vertically, 
 horizontally or diagonally.
 &quot;&quot;&quot;
 def solve(0, _m), do: [[]]
 def solve(n, m) do
 for done_queens &lt;- solve(n-1, m),
 avail_pos &lt;- (Enum.to_list(1..m) -- done_queens),
 safe_pos(avail_pos, done_queens, 1), 
 do: [avail_pos | done_queens]
 end

 defp safe_pos(_, [], _), do: true
 defp safe_pos(pos, [queen | queens], distance) do
 (pos != queen + distance) and 
 (pos != queen - distance) and 
 safe_pos(pos, queens, distance+1)
 end

end</code></pre>
Obviously, this is not the fastest algorithm to solve this problem (exponential complexity), but it shows how elegant the solution can be using Elixir’s comprehensions.
Sample output in iex:
<pre><code class="language-elixir">iex(47)&gt; c(&quot;queens.ex&quot;)
[Queens]
iex(48)&gt; :io.write Queens.solve(4, 4)
[[3,1,4,2],[2,4,1,3]]:ok
iex(49)&gt; :io.write Queens.solve(3, 4)
[[2,4,1],[1,4,2],[4,1,3],[3,1,4]]:ok
iex(50)&gt; :io.write Queens.solve(5, 5)
[[4,2,5,3,1],[3,5,2,4,1],[5,3,1,4,2],[4,1,3,5,2],[5,2,4,1,3],[1,4,2,5,3],[2,5,3,1,4],[1,3,5,2,4],[3,1,4,2,5],[2,4,1,3,5]]:ok
iex(51)&gt; :io.write Queens.solve(1, 5)
[[1],[2],[3],[4],[5]]:ok</code></pre>
Demo: <a href="http://elixirplayground.com?gist=688be58a64712a172878a58683ed0eda">http://elixirplayground.com?gist=688be58a64712a172878a58683ed0eda</a>
<h2 id="conclusion">Conclusion</h2>
Every day I learn more and more Elixir (and Erlang) and I can say that it’s among the few languages that managed to keep me interested and excited for more than 3-4 months (looking at you Scala and Python).

A naive recursive solution to the queens positioning problem using Elixir comprehensions.

Solve the eight queens problem with Elixir


## Problem

As you know I build this website using my custom site generator builder so the whole website is being re-created every time I change something. Similar tools are widely used by many people, like [Jekyll](https://jekyllrb.com/).

Then I use the [AWS S3 sync](http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html) command to update the public version of the site in my S3 bucket. The only problem with this is that since the local website is fully re-built every time it seems to be **newer** than the remote website in the S3 bucket (due to newer modification timestamps), resulting in extra uploads for all files. 

## Git Solution

The easiest solution to solve this problem is to use **git** to handle diff changes and then just pass along the modified files to the **aws s3 sync** command that will sync them against the remote S3 bucket.

So, long story short, assuming you have the site's directory git-versioned the following script will sync the directory with the remote S3 bucket, including adding new files, removing deleted files, etc.

```bash
#!/bin/bash
set -ex

FILES=()
for i in $( git status -s | sed 's/\s*[a-zA-Z?]\+ $.*$/\1/' ); do
    FILES+=( "$i" )
done
#echo "${FILES[@]}"

CMDS=()
for i in "${FILES[@]}"; do
    CMDS+=("--include=$i""*")
done
#echo "${CMDS[@]}"

echo "${CMDS[@]}" | xargs aws s3 sync . s3://www.lambrospetrou.com --dryrun --delete --exclude "*" 
```

**Important**

You have to remove the ```--dryrun``` option in order to actually apply the changes remotely, otherwise it will just fake them.

### Explanation

The important part of the above script is the ```--include``` and ```--exclude``` filters. The order of the filters **matters**, that's why we have the exclude first, and the includes last. In case the exclude was last nothing would be updated.

The two for-loops generate the required ```--include=FileX``` arguments, which are expanded using the ```"${CMDS[@]}"``` trick. Then **xargs** takes care of sending them as last arguments to the aws s3 sync command, also taking care of very long list of files that exceed the command line length limit.

In addition, I have to use ```git status``` instead of ```git diff``` otherwise new files will not be synced, since they are not part of the index tree.

## Conclusion

Using **git** along with **aws CLI** it's very easy to maintain my website and only upload the real **diff**, modified files, each time. One can imagine that this can be used in a much more advanced scenario with [Github webhooks](https://developer.github.com/webhooks/) integrated with [AWS CodePipeline](https://aws.amazon.com/codepipeline/) or any other CI tool that will release your website automatically.

## References

* [AWS S3 sync documentation](http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html)
* [git status documentation](https://git-scm.com/docs/git-status)


<h2 id="problem">Problem</h2>
As you know I build this website using my custom site generator builder so the whole website is being re-created every time I change something. Similar tools are widely used by many people, like <a href="https://jekyllrb.com/">Jekyll</a>.
Then I use the <a href="http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html">AWS S3 sync</a> command to update the public version of the site in my S3 bucket. The only problem with this is that since the local website is fully re-built every time it seems to be newer than the remote website in the S3 bucket (due to newer modification timestamps), resulting in extra uploads for all files. 
<h2 id="git-solution">Git Solution</h2>
The easiest solution to solve this problem is to use git to handle diff changes and then just pass along the modified files to the aws s3 sync command that will sync them against the remote S3 bucket.
So, long story short, assuming you have the site’s directory git-versioned the following script will sync the directory with the remote S3 bucket, including adding new files, removing deleted files, etc.
<pre><code class="language-bash">#!/bin/bash
set -ex

FILES=()
for i in $( git status -s | sed &#39;s/\s*[a-zA-Z?]\+ $.*$/\1/&#39; ); do
 FILES+=( &quot;$i&quot; )
done
#echo &quot;${FILES[@]}&quot;

CMDS=()
for i in &quot;${FILES[@]}&quot;; do
 CMDS+=(&quot;--include=$i&quot;&quot;*&quot;)
done
#echo &quot;${CMDS[@]}&quot;

echo &quot;${CMDS[@]}&quot; | xargs aws s3 sync . s3://www.lambrospetrou.com --dryrun --delete --exclude &quot;*&quot; </code></pre>
Important
You have to remove the <code>--dryrun</code> option in order to actually apply the changes remotely, otherwise it will just fake them.
<h3 id="explanation">Explanation</h3>
The important part of the above script is the <code>--include</code> and <code>--exclude</code> filters. The order of the filters matters, that’s why we have the exclude first, and the includes last. In case the exclude was last nothing would be updated.
The two for-loops generate the required <code>--include=FileX</code> arguments, which are expanded using the <code>&quot;${CMDS[@]}&quot;</code> trick. Then xargs takes care of sending them as last arguments to the aws s3 sync command, also taking care of very long list of files that exceed the command line length limit.
In addition, I have to use <code>git status</code> instead of <code>git diff</code> otherwise new files will not be synced, since they are not part of the index tree.
<h2 id="conclusion">Conclusion</h2>
Using git along with aws CLI it’s very easy to maintain my website and only upload the real diff, modified files, each time. One can imagine that this can be used in a much more advanced scenario with <a href="https://developer.github.com/webhooks/">Github webhooks</a> integrated with <a href="https://aws.amazon.com/codepipeline/">AWS CodePipeline</a> or any other CI tool that will release your website automatically.
<h2 id="references">References</h2>
<ul>
<li><a href="http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html">AWS S3 sync documentation</a></li>
<li><a href="https://git-scm.com/docs/git-status">git status documentation</a></li>
</ul>

An easy script to sync folders with S3 but only the updated, modified, files. I use git to properly handle changes.

AWS S3 sync - only modified files, using git status


Every now and then we have to change phones and transfer contacts from one device to another. It's a trivial thing to do nowadays with Android, iOS and Windows Phone, but try to do it on an old Nokia Symbian S40 phone. It's 2016 and this simple thing is just **fuxxing too complex**. And it's standards inconsistencies and Nokia to blame!

## Problem

I have my contacts in [Google Contacts](https://contacts.google.com), how they got there from an older S40 device is an even more fascinating story but let's leave it for another post. Now I just want to import these contacts into a [Nokia 301](http://www.gsmarena.com/nokia_301-5323.php), which runs Nokia Symbian S40, and yes some people still use this kind of phones, obviously not me :)

## Solution 1 - Nokia PC Suite

Nokia PC Suite was a nice tool by Nokia, **if and only if** it worked with your phone. In order to make it to recognise the device I had to install specific versions of the software (Nokia PC Suite version 7.1.180.94) and specific version of the Nokia Connectivity Cable driver (version 7.1.182.0).

OK, we are connected and we can manage the device through the PC suite. Now, what?

First of all, I needed to have the contacts in a form that can be imported somehow by the PC suite, so I exported all the contacts from Google to **.csv** and **.vcf** formats just in case one did not work.

There is a handy option in PC suite when you open the contacts section:

```
File => Import
```

which allows you to select any .csv, .vcf file to directly import into your phone. Of course this would be too easy :) so neither the .csv nor the .vcf import worked using the files from Google Export.

The problem with **.csv** is that Google's csv format follows a different structure compared to Nokia's PC suite so that's a **no-go**. The problem with **.vcf** is that this one file was containing ~1000 contacts but the import functionality was only importing 1 contact (the same happens when you send this .vcf file directly to the phone via Bluetooth).

After spending several hours going through myriads of forums with people asking for a solution I found [this article](http://blog.gluga.com/2009/11/import-google-contacts-to-nokia-pc.html) written in 2009 that provides a solution to split this single .vcf file into multiple .vcf files, one for each contact. The tool mentioned in the article, which I also used is [vCard Split](http://www.philipstorry.net/software/vcardsplit) by [Philip Storry](http://www.philipstorry.net/).

**Very important**

* In step 3 (see screenshot below) you have to select **Force version 2.1** otherwise Nokia PC suite will complain and fail to import the contacts. And also select/check the **Remove Type information** option because Google puts a TYPE section in each .vcf which is again not recognizable by PC suite.

![vCard Split screenshot](/articles/nokia-symbian-s40-contacts-transfer/vcard-screenshot.png "vCard Split screenshot")

Awesome, we have all our contacts splitted in a single .vcf file per contact. Now the last thing to do is just select all these .vcf files, COPY them (CTRL + C) and then PASTE them (CTRL + V) into the PC suite contacts section, or simply use the **File => Import** wizard and select all the .vcf files from the opened window.

Great, the tool seems to be importing the files and finishes successfully. **But**, there is a catch :) It seems that Nokia only cares about english/latin characters and any Greek character is converted into a weird symbol which makes your contacts **useless**. 

I searched for quite some time to find a way to bypass this limitation but it's not possible, so PC suite is out of the question. If you want Greek contact names, Nokia PC suite is a **BIG FAIL**.

## Solution 2 - Nokia Transfer app and an Android phone

To be honest this was the first solution I tried but it instantly failed so I went on with PC suite, until that one also failed and I returned to this one. This time I noticed that it was just a small step missing to make it work.

Nokia bundles with the Nokia 301 an application named **Tansfer** which you can also manually install on your own by downloading the **.jar/.jad** setup files and using PC suite to install them on the device. I manually installed the latest version **Transfer 1.0.11**.

This application's functionality is transferring data from **some** phones over to your device via **bluetooth**. I have a Motorolla Moto X and Moto G3 so I tried to use this and transfer the contacts directly from the Android phone to the Nokia phone. The problem was that I was always getting the error **This service is not supported by the phone** with both Androids, which was weird.

Then I went with PC suite route but we know how that ended up. So I decided to give another go on this simple method. This time, the magic was that I deleted the **bluetooth pairing** between the phones on both of them. 

The following steps did the job:

1. Turn on bluetooth on Moto G3
2. Turn on bluetooth on Nokia 301
3. On Nokia 301 go to bluetooth trusted devices and search for new one. Select the Moto G3 and accept the prompts in both phones, **checking/selecting** the option shown in Moto G3 which says **Allow Nokia 301 to read contact information**.
4. Open **Nokia Transfer** app on Nokia 301 (inside the Applications menu option)
5. Select **Add new device** and wait until Moto G3 appears and select it
6. Select Moto G3 from the populated list
7. Select **Import contacts**
8. Wait for the import to finish :)

And now I finally have all the contacts from Google into the Nokia 301.

## Conclusion

If you have a compatible device with Nokia Transfer app it is by far the easiest way to import your contacts into a new Nokia S40 phone, but remember to delete the pairing and then do it properly allowing access to contacts.

If you want to use PC Suite, unfortunately you cannot import contacts with Greek names, and most probably any contact name with non-ANSI characters.

**But please**, upgrade your phones to something modern, preferably Android :) 

## References

* [Import Google Contacts to Nokia PC Suite](http://blog.gluga.com/2009/11/import-google-contacts-to-nokia-pc.html)
* [vCard Split](http://www.philipstorry.net/software/vcardsplit)
* [Nokia Suite Support for Nokia 301 - Microsoft Answers](http://answers.microsoft.com/en-us/mobiledevices/forum/mdasha/nokia-suite-support-for-nokia-301/58a39104-d1ff-4ff1-91af-983ded51ea27)


Every now and then we have to change phones and transfer contacts from one device to another. It’s a trivial thing to do nowadays with Android, iOS and Windows Phone, but try to do it on an old Nokia Symbian S40 phone. It’s 2016 and this simple thing is just fuxxing too complex. And it’s standards inconsistencies and Nokia to blame!
<h2 id="problem">Problem</h2>
I have my contacts in <a href="https://contacts.google.com">Google Contacts</a>, how they got there from an older S40 device is an even more fascinating story but let’s leave it for another post. Now I just want to import these contacts into a <a href="http://www.gsmarena.com/nokia_301-5323.php">Nokia 301</a>, which runs Nokia Symbian S40, and yes some people still use this kind of phones, obviously not me :)
<h2 id="solution-1---nokia-pc-suite">Solution 1 - Nokia PC Suite</h2>
Nokia PC Suite was a nice tool by Nokia, if and only if it worked with your phone. In order to make it to recognise the device I had to install specific versions of the software (Nokia PC Suite version 7.1.180.94) and specific version of the Nokia Connectivity Cable driver (version 7.1.182.0).
OK, we are connected and we can manage the device through the PC suite. Now, what?
First of all, I needed to have the contacts in a form that can be imported somehow by the PC suite, so I exported all the contacts from Google to .csv and .vcf formats just in case one did not work.
There is a handy option in PC suite when you open the contacts section:
<pre><code>File =&gt; Import</code></pre>which allows you to select any .csv, .vcf file to directly import into your phone. Of course this would be too easy :) so neither the .csv nor the .vcf import worked using the files from Google Export.
The problem with .csv is that Google’s csv format follows a different structure compared to Nokia’s PC suite so that’s a no-go. The problem with .vcf is that this one file was containing ~1000 contacts but the import functionality was only importing 1 contact (the same happens when you send this .vcf file directly to the phone via Bluetooth).
After spending several hours going through myriads of forums with people asking for a solution I found <a href="http://blog.gluga.com/2009/11/import-google-contacts-to-nokia-pc.html">this article</a> written in 2009 that provides a solution to split this single .vcf file into multiple .vcf files, one for each contact. The tool mentioned in the article, which I also used is <a href="http://www.philipstorry.net/software/vcardsplit">vCard Split</a> by <a href="http://www.philipstorry.net/">Philip Storry</a>.
Very important
<ul>
<li>In step 3 (see screenshot below) you have to select Force version 2.1 otherwise Nokia PC suite will complain and fail to import the contacts. And also select/check the Remove Type information option because Google puts a TYPE section in each .vcf which is again not recognizable by PC suite.</li>
</ul>
<img src="/articles/nokia-symbian-s40-contacts-transfer/vcard-screenshot.png" alt="vCard Split screenshot" title="vCard Split screenshot"/>
Awesome, we have all our contacts splitted in a single .vcf file per contact. Now the last thing to do is just select all these .vcf files, COPY them (CTRL + C) and then PASTE them (CTRL + V) into the PC suite contacts section, or simply use the File =&gt; Import wizard and select all the .vcf files from the opened window.
Great, the tool seems to be importing the files and finishes successfully. But, there is a catch :) It seems that Nokia only cares about english/latin characters and any Greek character is converted into a weird symbol which makes your contacts useless. 
I searched for quite some time to find a way to bypass this limitation but it’s not possible, so PC suite is out of the question. If you want Greek contact names, Nokia PC suite is a BIG FAIL.
<h2 id="solution-2---nokia-transfer-app-and-an-android-phone">Solution 2 - Nokia Transfer app and an Android phone</h2>
To be honest this was the first solution I tried but it instantly failed so I went on with PC suite, until that one also failed and I returned to this one. This time I noticed that it was just a small step missing to make it work.
Nokia bundles with the Nokia 301 an application named Tansfer which you can also manually install on your own by downloading the .jar/.jad setup files and using PC suite to install them on the device. I manually installed the latest version Transfer 1.0.11.
This application’s functionality is transferring data from some phones over to your device via bluetooth. I have a Motorolla Moto X and Moto G3 so I tried to use this and transfer the contacts directly from the Android phone to the Nokia phone. The problem was that I was always getting the error This service is not supported by the phone with both Androids, which was weird.
Then I went with PC suite route but we know how that ended up. So I decided to give another go on this simple method. This time, the magic was that I deleted the bluetooth pairing between the phones on both of them. 
The following steps did the job:
<ol>
<li>Turn on bluetooth on Moto G3</li>
<li>Turn on bluetooth on Nokia 301</li>
<li>On Nokia 301 go to bluetooth trusted devices and search for new one. Select the Moto G3 and accept the prompts in both phones, checking/selecting the option shown in Moto G3 which says Allow Nokia 301 to read contact information.</li>
<li>Open Nokia Transfer app on Nokia 301 (inside the Applications menu option)</li>
<li>Select Add new device and wait until Moto G3 appears and select it</li>
<li>Select Moto G3 from the populated list</li>
<li>Select Import contacts</li>
<li>Wait for the import to finish :)</li>
</ol>
And now I finally have all the contacts from Google into the Nokia 301.
<h2 id="conclusion">Conclusion</h2>
If you have a compatible device with Nokia Transfer app it is by far the easiest way to import your contacts into a new Nokia S40 phone, but remember to delete the pairing and then do it properly allowing access to contacts.
If you want to use PC Suite, unfortunately you cannot import contacts with Greek names, and most probably any contact name with non-ANSI characters.
But please, upgrade your phones to something modern, preferably Android :) 
<h2 id="references">References</h2>
<ul>
<li><a href="http://blog.gluga.com/2009/11/import-google-contacts-to-nokia-pc.html">Import Google Contacts to Nokia PC Suite</a></li>
<li><a href="http://www.philipstorry.net/software/vcardsplit">vCard Split</a></li>
<li><a href="http://answers.microsoft.com/en-us/mobiledevices/forum/mdasha/nokia-suite-support-for-nokia-301/58a39104-d1ff-4ff1-91af-983ded51ea27">Nokia Suite Support for Nokia 301 - Microsoft Answers</a></li>
</ul>

A tutorial on how to copy your contacts to an old Nokia S40 phone.

Transfer your contacts to an old Nokia Symbian S40 mobile phone


In this small tutorial I provide a code snippet that allows you to update a record set in [Route53](https://aws.amazon.com/route53/) with the public IP of an [EC2](https://aws.amazon.com/ec2/) instance programmatically.

## Why? Just use an Elastic Load Balancer!

[Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) is an awesome service that handles load balancing of the traffic to your instances very well. But, it comes with a price, literally. An ELB charges a small amount for every hour running, pretty much like a **T2.small** instance (as of time of writing). This is dirty-cheap when you have tens or thousands of instances behind it, but for 1-2 **T2.nano** instances it might seem overkill.

I play around a lot with AWS and I create small projects, websites mostly, where a single node is more than enough to handle all the load, thus paying for an additional load balancer is too much.

Moreover, there are many people that have use cases where ELB is not the right solution and they prefer to have **Route53** act as their load balancer. Loggly team wrote a [very nice article](https://www.loggly.com/blog/why-aws-route-53-over-elastic-load-balancing/) describing this approach and I recommend it for anyone interested in real-world advantages.

## Alternatives?

There are several custom solutions to avoid using an ELB. Most of them take advantage of [Elastic IPs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html) and Route53 API to programmatically update record sets.

### Solution 1 - Elastic IP only

This solution is as simple as possible and only applies to single instance applications. When the EC2 instance boots you provide a user data script which will associate your Elastic IP (you should create this before booting the instance) with the instance.

Find more information for the exact CLI command at the [official EC2 CLI reference](http://docs.aws.amazon.com/cli/latest/reference/ec2/associate-address.html#examples).

I recently discovered that this is how **single-node** configuration of Elastic Beanstalk is implemented (very similar).

### Solution 2 - Route 53 only

This solution requires a bit more work from you in terms of scripting but it is more flexible than having an Elastic IP for each instance. Inside the user data script of your launch-configuration you have full access to the AWS CLI. Therefore, you can pretty much do anything! 

Let's assume that you have an application at **test.lambrospetrou.com** served by a **single** instance. You want to programmatically update the **A** record of this record set to point to any new instance being created by your **max-1-min-1-desired-1** auto-scaling group.

It turns out it is very simple to do :)

First of all we need to find the public IP of the running instance. AWS provides several [metadata information to every EC2 instance](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-retrieval) and public IP is one of them.

```bash
curl http://169.254.169.254/latest/meta-data/public-ipv4
```

The next step is to update the **test.lambrospetrou.com** record set. It is pretty easy to navigate through the [Route53 - 'change-resource-record-sets' documentation](http://docs.aws.amazon.com/cli/latest/reference/route53/change-resource-record-sets.html) which we find that the JSON we need is pretty much the following:

```json
{
  "Comment": "Update the A record set",
  "Changes": [
    {
      "Action": "UPSERT",
      "ResourceRecordSet": {
        "Name": "test.lambrospetrou.com",
        "Type": "A",
        "TTL": 300,
        "ResourceRecords": [
          {
            "Value": "127.0.0.1"
          }
        ]
      }
    }
  ]
}
```

In the above JSON you can see that we want to do an **UPSERT** (update) for the **test.lambrospetrou.com** record set, with type **A** since we want to point to an IP address, and with value 127.0.0.1.

In order to do the record set update you need the following command (assume that the above JSON is the content of a file named **update-route53-A.json**):

```bash
aws route53 change-resource-record-sets --hosted-zone-id "$HOSTED_ZONE_ID" --change-batch file://./update-route53-A.json
```

In the above command you have to put the correct **Hosted zone id** where the record set resides. You can find this in the [Route53 console](https://console.aws.amazon.com/route53/) or with the following command:

```bash
aws route53 list-hosted-zones-by-name
```

If you try and play with the above command you will notice that the update is pretty much instant, which means that as soon as the instance is up and running, your application will be available using the domain.

There is one variation of the above command which accepts a string instead of a file, which makes it easier to use through scripting. The only difference is that the JSON we examined above needs to be enclosed in a field named **ChangeBatch**:

```json
{ "ChangeBatch": { /*INPUT_JSON_LIKE_BEFORE*/ } }
```

and the command to use this JSON string is as follows:

```bash
aws route53 change-resource-record-sets --hosted-zone-id "$HOSTED_ZONE_ID" --cli-input-json "$INPUT_JSON_STR"
```

To summarise, these are the only two commands we need to do what we want:

1. Get the public IP of the running instance

    ```bash
    curl http://169.254.169.254/latest/meta-data/public-ipv4
    ```

2. Update the Route53 record set

    ```bash
    aws route53 change-resource-record-sets --hosted-zone-id "$HOSTED_ZONE_ID" --cli-input-json "$INPUT_JSON_STR"
    ```

Of course you will want to do some more scripting to replace the **127.0.0.1** value with the proper IP and programmatically find the hosted zone id of your domain. The following snippet is a quick/hacky way of accomplishing this. Feel free to use it if you are bored to come up with a better one :)

```bash
#!/bin/sh

if [ -z "$1" ]; then 
    echo "IP not given...trying EC2 metadata...";
    IP=$( curl http://169.254.169.254/latest/meta-data/public-ipv4 )  
else 
    IP="$1" 
fi 
echo "IP to update: $IP"

HOSTED_ZONE_ID=$( aws route53 list-hosted-zones-by-name | grep -B 1 -e "lambrospetrou.com" | sed 's/.*hostedzone\/$[A-Za-z0-9]*$\".*/\1/' | head -n 1 )
echo "Hosted zone being modified: $HOSTED_ZONE_ID"

INPUT_JSON=$( cat ./update-route53-A.json | sed "s/127\.0\.0\.1/$IP/" )

# http://docs.aws.amazon.com/cli/latest/reference/route53/change-resource-record-sets.html
# We want to use the string variable command so put the file contents (batch-changes file) in the following JSON
INPUT_JSON="{ \"ChangeBatch\": $INPUT_JSON }"

aws route53 change-resource-record-sets --hosted-zone-id "$HOSTED_ZONE_ID" --cli-input-json "$INPUT_JSON"
```

As an additional note, the above example updates the A record with a single IP. You can easily adapt the script to retrieve the current IPs of the record set and append the new one to them. This way you can even achieve simple round robin load balancing between nodes exploiting Route53's weighted routing. Imagination is the limit to what you can achieve :)

### Solution 3 - Route53 & Lambda

Another interesting and even more flexible way of achieving what I explained above, and way much more, is utilising [AWS Lambda](https://aws.amazon.com/lambda/) functions. The combination of Lambda with events and Route53 is **super-powerful** and can implement very complex configuration updates that are usually very difficult.

A lot of people are already playing with Lambda for doing network configuration changes triggered by **autoscaling**. For example, instead of doing the update of the Route53 record set in the user data section of the EC2 launch-configuration, you could setup a Lambda function to be invoked when a new instance has been created (or terminated) and do your changes using the official AWS SDKs available in Lambda functions.

Many AWS tutorials are available in the [AWS blogs](https://aws.amazon.com/blogs/) but you can find some in the References section below.

## Conclusion

The above solution is by no means comprehensive or suitable for everyone and every use-case. Most of the time you will **want** to use Elastic Load Balancing which provides cross-AZ balancing, monitoring, and transparent load balancing in front of several auto-scaling groups.

In some cases though you want to keep it simple or **cheap**, and this is where these solutions are preferred!

Feel free to contact me for any mistakes you find in the snippets above or if you have a better solution for the aforementioned problem.

Happy AWS Clouding :)

## References

* [Why Loggly Chose Amazon Route 53 over Elastic Load Balancing](https://www.loggly.com/blog/why-aws-route-53-over-elastic-load-balancing/)
* [Using AWS Lambda with Auto Scaling Lifecycle Hooks](https://aws.amazon.com/blogs/compute/using-aws-lambda-with-auto-scaling-lifecycle-hooks/)
* [Building a Dynamic DNS for Route 53 using CloudWatch Events and Lambda](https://aws.amazon.com/blogs/compute/building-a-dynamic-dns-for-route-53-using-cloudwatch-events-and-lambda/)
* [Auto Scaling Lifecycle Hooks](http://docs.aws.amazon.com/autoscaling/latest/userguide/lifecycle-hooks.html)
* [Elastic IP Addresses](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html)
* [Official EC2 CLI - Associate Address reference](http://docs.aws.amazon.com/cli/latest/reference/ec2/associate-address.html#examples)


In this small tutorial I provide a code snippet that allows you to update a record set in <a href="https://aws.amazon.com/route53/">Route53</a> with the public IP of an <a href="https://aws.amazon.com/ec2/">EC2</a> instance programmatically.
<h2 id="why-just-use-an-elastic-load-balancer">Why? Just use an Elastic Load Balancer!</h2>
<a href="https://aws.amazon.com/elasticloadbalancing/">Elastic Load Balancing</a> is an awesome service that handles load balancing of the traffic to your instances very well. But, it comes with a price, literally. An ELB charges a small amount for every hour running, pretty much like a T2.small instance (as of time of writing). This is dirty-cheap when you have tens or thousands of instances behind it, but for 1-2 T2.nano instances it might seem overkill.
I play around a lot with AWS and I create small projects, websites mostly, where a single node is more than enough to handle all the load, thus paying for an additional load balancer is too much.
Moreover, there are many people that have use cases where ELB is not the right solution and they prefer to have Route53 act as their load balancer. Loggly team wrote a <a href="https://www.loggly.com/blog/why-aws-route-53-over-elastic-load-balancing/">very nice article</a> describing this approach and I recommend it for anyone interested in real-world advantages.
<h2 id="alternatives">Alternatives?</h2>
There are several custom solutions to avoid using an ELB. Most of them take advantage of <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html">Elastic IPs</a> and Route53 API to programmatically update record sets.
<h3 id="solution-1---elastic-ip-only">Solution 1 - Elastic IP only</h3>
This solution is as simple as possible and only applies to single instance applications. When the EC2 instance boots you provide a user data script which will associate your Elastic IP (you should create this before booting the instance) with the instance.
Find more information for the exact CLI command at the <a href="http://docs.aws.amazon.com/cli/latest/reference/ec2/associate-address.html#examples">official EC2 CLI reference</a>.
I recently discovered that this is how single-node configuration of Elastic Beanstalk is implemented (very similar).
<h3 id="solution-2---route-53-only">Solution 2 - Route 53 only</h3>
This solution requires a bit more work from you in terms of scripting but it is more flexible than having an Elastic IP for each instance. Inside the user data script of your launch-configuration you have full access to the AWS CLI. Therefore, you can pretty much do anything! 
Let’s assume that you have an application at test.lambrospetrou.com served by a single instance. You want to programmatically update the A record of this record set to point to any new instance being created by your max-1-min-1-desired-1 auto-scaling group.
It turns out it is very simple to do :)
First of all we need to find the public IP of the running instance. AWS provides several <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-retrieval">metadata information to every EC2 instance</a> and public IP is one of them.
<pre><code class="language-bash">curl http://169.254.169.254/latest/meta-data/public-ipv4</code></pre>
The next step is to update the test.lambrospetrou.com record set. It is pretty easy to navigate through the <a href="http://docs.aws.amazon.com/cli/latest/reference/route53/change-resource-record-sets.html">Route53 - ‘change-resource-record-sets’ documentation</a> which we find that the JSON we need is pretty much the following:
<pre><code class="language-json">{
 &quot;Comment&quot;: &quot;Update the A record set&quot;,
 &quot;Changes&quot;: [
 {
 &quot;Action&quot;: &quot;UPSERT&quot;,
 &quot;ResourceRecordSet&quot;: {
 &quot;Name&quot;: &quot;test.lambrospetrou.com&quot;,
 &quot;Type&quot;: &quot;A&quot;,
 &quot;TTL&quot;: 300,
 &quot;ResourceRecords&quot;: [
 {
 &quot;Value&quot;: &quot;127.0.0.1&quot;
 }
 ]
 }
 }
 ]
}</code></pre>
In the above JSON you can see that we want to do an UPSERT (update) for the test.lambrospetrou.com record set, with type A since we want to point to an IP address, and with value 127.0.0.1.
In order to do the record set update you need the following command (assume that the above JSON is the content of a file named update-route53-A.json):
<pre><code class="language-bash">aws route53 change-resource-record-sets --hosted-zone-id &quot;$HOSTED_ZONE_ID&quot; --change-batch file://./update-route53-A.json</code></pre>
In the above command you have to put the correct Hosted zone id where the record set resides. You can find this in the <a href="https://console.aws.amazon.com/route53/">Route53 console</a> or with the following command:
<pre><code class="language-bash">aws route53 list-hosted-zones-by-name</code></pre>
If you try and play with the above command you will notice that the update is pretty much instant, which means that as soon as the instance is up and running, your application will be available using the domain.
There is one variation of the above command which accepts a string instead of a file, which makes it easier to use through scripting. The only difference is that the JSON we examined above needs to be enclosed in a field named ChangeBatch:
<pre><code class="language-json">{ &quot;ChangeBatch&quot;: { /*INPUT_JSON_LIKE_BEFORE*/ } }</code></pre>
and the command to use this JSON string is as follows:
<pre><code class="language-bash">aws route53 change-resource-record-sets --hosted-zone-id &quot;$HOSTED_ZONE_ID&quot; --cli-input-json &quot;$INPUT_JSON_STR&quot;</code></pre>
To summarise, these are the only two commands we need to do what we want:
<ol>
<li>Get the public IP of the running instance
<pre><code class="language-bash"> curl http://169.254.169.254/latest/meta-data/public-ipv4</code></pre>
</li>
<li>Update the Route53 record set
<pre><code class="language-bash"> aws route53 change-resource-record-sets --hosted-zone-id &quot;$HOSTED_ZONE_ID&quot; --cli-input-json &quot;$INPUT_JSON_STR&quot;</code></pre>
</li>
</ol>
Of course you will want to do some more scripting to replace the 127.0.0.1 value with the proper IP and programmatically find the hosted zone id of your domain. The following snippet is a quick/hacky way of accomplishing this. Feel free to use it if you are bored to come up with a better one :)
<pre><code class="language-bash">#!/bin/sh

if [ -z &quot;$1&quot; ]; then 
 echo &quot;IP not given...trying EC2 metadata...&quot;;
 IP=$( curl http://169.254.169.254/latest/meta-data/public-ipv4 ) 
else 
 IP=&quot;$1&quot; 
fi 
echo &quot;IP to update: $IP&quot;

HOSTED_ZONE_ID=$( aws route53 list-hosted-zones-by-name | grep -B 1 -e &quot;lambrospetrou.com&quot; | sed &#39;s/.*hostedzone\/$[A-Za-z0-9]*$\&quot;.*/\1/&#39; | head -n 1 )
echo &quot;Hosted zone being modified: $HOSTED_ZONE_ID&quot;

INPUT_JSON=$( cat ./update-route53-A.json | sed &quot;s/127\.0\.0\.1/$IP/&quot; )

# http://docs.aws.amazon.com/cli/latest/reference/route53/change-resource-record-sets.html
# We want to use the string variable command so put the file contents (batch-changes file) in the following JSON
INPUT_JSON=&quot;{ \&quot;ChangeBatch\&quot;: $INPUT_JSON }&quot;

aws route53 change-resource-record-sets --hosted-zone-id &quot;$HOSTED_ZONE_ID&quot; --cli-input-json &quot;$INPUT_JSON&quot;</code></pre>
As an additional note, the above example updates the A record with a single IP. You can easily adapt the script to retrieve the current IPs of the record set and append the new one to them. This way you can even achieve simple round robin load balancing between nodes exploiting Route53’s weighted routing. Imagination is the limit to what you can achieve :)
<h3 id="solution-3---route53--lambda">Solution 3 - Route53 &amp; Lambda</h3>
Another interesting and even more flexible way of achieving what I explained above, and way much more, is utilising <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> functions. The combination of Lambda with events and Route53 is super-powerful and can implement very complex configuration updates that are usually very difficult.
A lot of people are already playing with Lambda for doing network configuration changes triggered by autoscaling. For example, instead of doing the update of the Route53 record set in the user data section of the EC2 launch-configuration, you could setup a Lambda function to be invoked when a new instance has been created (or terminated) and do your changes using the official AWS SDKs available in Lambda functions.
Many AWS tutorials are available in the <a href="https://aws.amazon.com/blogs/">AWS blogs</a> but you can find some in the References section below.
<h2 id="conclusion">Conclusion</h2>
The above solution is by no means comprehensive or suitable for everyone and every use-case. Most of the time you will want to use Elastic Load Balancing which provides cross-AZ balancing, monitoring, and transparent load balancing in front of several auto-scaling groups.
In some cases though you want to keep it simple or cheap, and this is where these solutions are preferred!
Feel free to contact me for any mistakes you find in the snippets above or if you have a better solution for the aforementioned problem.
Happy AWS Clouding :)
<h2 id="references">References</h2>
<ul>
<li><a href="https://www.loggly.com/blog/why-aws-route-53-over-elastic-load-balancing/">Why Loggly Chose Amazon Route 53 over Elastic Load Balancing</a></li>
<li><a href="https://aws.amazon.com/blogs/compute/using-aws-lambda-with-auto-scaling-lifecycle-hooks/">Using AWS Lambda with Auto Scaling Lifecycle Hooks</a></li>
<li><a href="https://aws.amazon.com/blogs/compute/building-a-dynamic-dns-for-route-53-using-cloudwatch-events-and-lambda/">Building a Dynamic DNS for Route 53 using CloudWatch Events and Lambda</a></li>
<li><a href="http://docs.aws.amazon.com/autoscaling/latest/userguide/lifecycle-hooks.html">Auto Scaling Lifecycle Hooks</a></li>
<li><a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html">Elastic IP Addresses</a></li>
<li><a href="http://docs.aws.amazon.com/cli/latest/reference/ec2/associate-address.html#examples">Official EC2 CLI - Associate Address reference</a></li>
</ul>

A cool snippet showing how to update a record set in Route53 with a new IP value. This is very helpful in case you want to have an application running on a few instances but you do not want the charge overheads of an Elastic Load Balancer.

Update Route53 record set with EC2 instance public IP for a DIY load balancer


A few months ago I attended a talk given by Monotype's [Steve Matteson](http://monotype.de/studio/steve-matteson) who designed awesome fonts like **Open Sans**, Android's **Droid family** and Microsoft's **Segoe family** among others, and by [Mark Boulton](http://markboulton.co.uk/) who is now Design Director at [Monotype](http://monotype.com/) and founder of [Mark Boulton Design](http://www.markboultondesign.com/), which was acquired by Monotype :)

The talk was pretty great and as you can imagine from their background, it was about Typefaces and their importance in a brand's or a product's value. **Yes**, I am a software engineer but I **do** care about the design and the aesthetics of my work! As much as I hate Apple, **Steve Jobs** was the leading exemplar of the fusion between technology and design, and will always be one of my role models.

Anyway, the one thing that stood out and impressed me the most out of the talk was Mark's speech about **Marginal Degradation**. I found it so basic and simple but at the same time so powerful that it just sticked to my mind. After a few days I decided to read more about it and found a [related article in Mark's blog](http://markboulton.co.uk/journal/marginal-degredation) which is basically a scripted version of what he said. I always paid attention to the smallest detail, but after reading and hearing about this I just made it a habit! I will give you a brief summary below but I encourage you to read the whole article.

![Banana vs Human DNA](/articles/banana-or-human-marginal-degradation/banana-small.jpg "Banana vs Human DNA")

> Here’s a few fun facts… - Gorillas share 98.4% of our DNA - Goldfish share 68% - Bananas share 50%
>
> Bananas. Are 50% the same as us.

**Read** the above quote, then read it **again**, and then **again one last time!**

Only 1.6% of DNA differentiates us from gorillas and we are half-way to becoming bananas! Don't get all fancy science genius with me now, I don't know how much really that is in DNA world, but the fact still holds. Even the slightest marginal degradation of our DNA can make us inferior.

Now consider this to whatever you do. I write code as a living, you might be writing literature, or you might be designing the next website for the most awesome startup ever. You **have to** pay attention to detail, even the smallest one! Do not let anyone tell you "Just do it quickly for now, the manager wants it urgently. We can revisit this later." **No**, you will **not** revisit it later and you will **not** fix this little sucker. It is there and it will stay there, degrading and sucking out the value and quality of your product.

> Your brand or design is supposed to be a human, but people perceive it as a gorilla. Or a banana.

You have to really think the consequences of everything you sacrifice by not giving 100% of yourself in doing something. It might seem like a tiny thing at the moment, but if you take into account **all** those little tiny things you sacrificed over the years, you will find out how much you deviated from your original goal. 

Nobody wants his product or brand to be seen as a banana, when it was supposed to be a human!

## Takeouts

* Pay attention to the details!
* Do not sacrifice quality for a moment's weakness or lack of time!
* Always have in mind the **original** goal/target of what you want to achieve, otherwise every little deviation might lead to something entirely different!

## References

* [The difference between a goldfish and a human](http://markboulton.co.uk/journal/marginal-degredation)

A few months ago I attended a talk given by Monotype’s <a href="http://monotype.de/studio/steve-matteson">Steve Matteson</a> who designed awesome fonts like Open Sans, Android’s Droid family and Microsoft’s Segoe family among others, and by <a href="http://markboulton.co.uk/">Mark Boulton</a> who is now Design Director at <a href="http://monotype.com/">Monotype</a> and founder of <a href="http://www.markboultondesign.com/">Mark Boulton Design</a>, which was acquired by Monotype :)
The talk was pretty great and as you can imagine from their background, it was about Typefaces and their importance in a brand’s or a product’s value. Yes, I am a software engineer but I do care about the design and the aesthetics of my work! As much as I hate Apple, Steve Jobs was the leading exemplar of the fusion between technology and design, and will always be one of my role models.
Anyway, the one thing that stood out and impressed me the most out of the talk was Mark’s speech about Marginal Degradation. I found it so basic and simple but at the same time so powerful that it just sticked to my mind. After a few days I decided to read more about it and found a <a href="http://markboulton.co.uk/journal/marginal-degredation">related article in Mark’s blog</a> which is basically a scripted version of what he said. I always paid attention to the smallest detail, but after reading and hearing about this I just made it a habit! I will give you a brief summary below but I encourage you to read the whole article.
<img src="/articles/banana-or-human-marginal-degradation/banana-small.jpg" alt="Banana vs Human DNA" title="Banana vs Human DNA"/>
<blockquote>
Here’s a few fun facts… - Gorillas share 98.4% of our DNA - Goldfish share 68% - Bananas share 50%
Bananas. Are 50% the same as us.
</blockquote>
Read the above quote, then read it again, and then again one last time!
Only 1.6% of DNA differentiates us from gorillas and we are half-way to becoming bananas! Don’t get all fancy science genius with me now, I don’t know how much really that is in DNA world, but the fact still holds. Even the slightest marginal degradation of our DNA can make us inferior.
Now consider this to whatever you do. I write code as a living, you might be writing literature, or you might be designing the next website for the most awesome startup ever. You have to pay attention to detail, even the smallest one! Do not let anyone tell you “Just do it quickly for now, the manager wants it urgently. We can revisit this later.” No, you will not revisit it later and you will not fix this little sucker. It is there and it will stay there, degrading and sucking out the value and quality of your product.
<blockquote>
Your brand or design is supposed to be a human, but people perceive it as a gorilla. Or a banana.
</blockquote>
You have to really think the consequences of everything you sacrifice by not giving 100% of yourself in doing something. It might seem like a tiny thing at the moment, but if you take into account all those little tiny things you sacrificed over the years, you will find out how much you deviated from your original goal. 
Nobody wants his product or brand to be seen as a banana, when it was supposed to be a human!
<h2 id="takeouts">Takeouts</h2>
<ul>
<li>Pay attention to the details!</li>
<li>Do not sacrifice quality for a moment’s weakness or lack of time!</li>
<li>Always have in mind the original goal/target of what you want to achieve, otherwise every little deviation might lead to something entirely different!</li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><a href="http://markboulton.co.uk/journal/marginal-degredation">The difference between a goldfish and a human</a></li>
</ul>

An article explaining how marginal degradation ruins your goals.

Banana or Human, and Marginal Degradation


This is going to probably be the shortest article I wrote, or will write!

Have you ever wanted to open the right click menu on something without lifting your hands from the keyboard? I did a lot of times, especially when navigating through videos wanting to **open** them **with** the non-default application (which can be done with the **Open with** option in the right click context menu).

Well, you can do it with the following keyboard shortcut:

```bash
Shift + F10
```

**Warning:** It does not work in **all** the applications, but it does in the file manager I use which is enough for me :)

Alternatively, you can set keyboard shortcuts to trigger mouse events including right click, but this is out of this article's scope.

This is going to probably be the shortest article I wrote, or will write!
Have you ever wanted to open the right click menu on something without lifting your hands from the keyboard? I did a lot of times, especially when navigating through videos wanting to open them with the non-default application (which can be done with the Open with option in the right click context menu).
Well, you can do it with the following keyboard shortcut:
<pre><code class="language-bash">Shift + F10</code></pre>
Warning: It does not work in all the applications, but it does in the file manager I use which is enough for me :)
Alternatively, you can set keyboard shortcuts to trigger mouse events including right click, but this is out of this article’s scope.

A shortcut that opens context menu, known as right-click, in most applications.

Keyboard shortcut for opening context menu


A few months ago I started migrating all my projects and websites into AWS. The last thing standing was my old wordpress blog (hosted at **mastergenius.net**, not alive anymore). Today, I decided to make a final backup and create a static version out of it and ditch the VPS I have running just for that.

It turned out to be extremely easy :) 

## Backup all content

Apart from serving the content as static website I wanted to have the posts in a form that can be easily imported later back in Wordpress and also a format which is easy to parse and read using a text editor.

I just wanted the **text**, which is the heart of any article anyway, so I just did the usual **export** feature of Wordpress as described in the [official export documentation](https://en.support.wordpress.com/export/).

A full folder copy is recommended too, you know just in case I want to revisit any files, images or code!

## Serve as static website

There are a lot of ways to dump an active wordpress installation and several plugins that allow you to do this conversion easily. I just took the simplest way and used the well-known **wget** Linux command line tool (original Quora question link with the command is in the References section).

```bash
wget -k -K  -E -r -l 10 -p -N -F --restrict-file-names=windows -nH http://active-wordpress.domain.com
```

After executing the above command I deleted all the unnecessary files, stripping down the whole site in some MBs.

I just uploaded the static files to [Amazon Simple Storage Service - S3](http://aws.amazon.com/s3/) and that's all!

Zero-cost hosting :)

## References

* [How do you export a WordPress site to a static HTML?](https://www.quora.com/How-do-you-export-a-WordPress-site-to-a-static-HTML)
* [Wordpress export documentation](https://en.support.wordpress.com/export/)

A few months ago I started migrating all my projects and websites into AWS. The last thing standing was my old wordpress blog (hosted at mastergenius.net, not alive anymore). Today, I decided to make a final backup and create a static version out of it and ditch the VPS I have running just for that.
It turned out to be extremely easy :) 
<h2 id="backup-all-content">Backup all content</h2>
Apart from serving the content as static website I wanted to have the posts in a form that can be easily imported later back in Wordpress and also a format which is easy to parse and read using a text editor.
I just wanted the text, which is the heart of any article anyway, so I just did the usual export feature of Wordpress as described in the <a href="https://en.support.wordpress.com/export/">official export documentation</a>.
A full folder copy is recommended too, you know just in case I want to revisit any files, images or code!
<h2 id="serve-as-static-website">Serve as static website</h2>
There are a lot of ways to dump an active wordpress installation and several plugins that allow you to do this conversion easily. I just took the simplest way and used the well-known wget Linux command line tool (original Quora question link with the command is in the References section).
<pre><code class="language-bash">wget -k -K -E -r -l 10 -p -N -F --restrict-file-names=windows -nH http://active-wordpress.domain.com</code></pre>
After executing the above command I deleted all the unnecessary files, stripping down the whole site in some MBs.
I just uploaded the static files to <a href="http://aws.amazon.com/s3/">Amazon Simple Storage Service - S3</a> and that’s all!
Zero-cost hosting :)
<h2 id="references">References</h2>
<ul>
<li><a href="https://www.quora.com/How-do-you-export-a-WordPress-site-to-a-static-HTML">How do you export a WordPress site to a static HTML?</a></li>
<li><a href="https://en.support.wordpress.com/export/">Wordpress export documentation</a></li>
</ul>

An easy way to backup your Wordpress installation and serve it as a static website.

Backup your Wordpress and serve it as a static website


I decided to play around with [Typescript](https://www.typescriptlang.org/) instead of [BabelJS](https://babeljs.io/), which I already tried in the past, for this weekend's project. 

## Reasons for Typescript

During the last year I explored several Javascript **languages/flavors** (transpilers, etc.). If you read my previous articles you should know that I also like [Dart](https://www.dartlang.org/). I started using it in 2014 along with [AngularDart](https://github.com/angular/angular.dart) throughout its journey to the stable version 1.0, always going through refactorings and fixing breaking changes. But Dart made me feel again productive and gave me a sensible platform to do front-end development. Unfortunately, it did not pick up as many fans as Google (and me) wanted so it was kinda left behind.

I really believe that the future lies in languages that compile to Javascript or even something else, **WebAssembly** maybe? ([WebAssembly homepage](https://webassembly.github.io/) and [Article](https://medium.com/javascript-scene/what-is-webassembly-the-dawn-of-a-new-era-61256ec5a8f6#.pb2824qir) )

I already gave a go to Babel, so this time it was time for Typescript. Why choose Typescript and not stay with Babel is justified by the points below.

* I love static typed languages :) and Typescript is much closer to that rather than JS
* I prefer compilation errors much more than runtime exceptions
* I believe it will be alive for at least 3 more years since it is backed by **Microsoft** and **Google** and is used as the main language for **Angular 2**
* Better IDE support (if I decide to use one)

## Project

I wanted to not spend more than 1-2 days so I have chosen something super simple to implement. I decided to develop a **numerology** calculator of your life path number :) Fancy stuff!

[Calculate your life path number at the demo page](/articles/webdev-typescript-webpack-react-numerology/demo/)

## Build Process

My main goal with the project was to come up with a build process using **Typescript** with **ES6**, **Webpack** and **ReactJS**. I also wanted to use **Autoprefixer** and **SASS** so I added **Gulp** to the mix too, but I tried to keep it at a bare minimum.

The final process is explained in the following sections.

### Code

I wanted from the beginning to consolidate the code building into one tool, and the decision was pretty easy since [Webpack](http://webpack.github.io/) is the big winner (at least for now).

The requirements are simple:

* Write my code in Typescript
* Use ES6 features
* Support browsers that only have ES5 support
* As a bonus I also wrote the gulpfile in Typescript and used a hack-around to use it to actually compile the Typescript code :)

For **ReactJS** I needed to install **react** and **react-dom** along with their typings to allow Typescript to resolve the React types. Pretty smooth process and I also found out something I did not know before. With webpack I can use the npm modules during development but **do not** bundle them along with my app code to keep the footprint small. By specifying that these modules are external I just need to use **script** tags in my HTML to import the public ReactJS libraries and take advantage of CDNs and local cached versions.

```bash
npm install typescript typings react-dom react --save-dev
typings install dt~react dt~react-dom --global --save
```

In order to be able to use some of the ES6 features, like **Object.assign()**, I had to include [es5-shim](https://cdnjs.com/libraries/es5-shim) and [es6-shim](https://cdnjs.com/libraries/es6-shim) in my HTML code right before I imported the Typescript compiled bundled which targets **ES5**. I also needed to install the typings for ES6 in order to stop Typescript compilation errors when using ES6 features in the code.

```bash
typings install dt~es6-shim --global --save
```

So, the Typescript compiler will use the typings and compile the code into ES5-valid Javascript and then Webpack will take all the files and output 2 bundles. I like to have separation between the app code which is Typescript and other JS code I might have, so I compile a bundle for the Typescript code **ts.bundle.js** and one for normal JS-code **js.bundle.js**.

**Use Typescript for the Gulpfile**

I could have used regular ES5 Javascript to write the Gulpfile but I wanted to play more with Typescript. I spend some time to be honest searching for solutions for this and I ended up following [this awesome guide](https://medium.com/@pleerock/create-a-gulpfile-and-write-gulp-tasks-using-typescript-f08edebcac57#.55q6zomio) adapting it where needed.

### SASS and Build

No surprises here! Gulp is used to clean the build directory, to use [Autoprefixer](https://github.com/postcss/autoprefixer) to prefix new/useful CSS with browser-prefixes and to build the final CSS using [SASS](http://sass-lang.com/).

## Conclusion

I spend some time back and forth with the Typescript documentation, expected since this is the first time I tried it, but I liked it. I am going to stick with this for a while until I find something much more productive and powerful, [ScalaJS](https://www.scala-js.org/) might be that one  :)

Finally, I liked ReactJS too, and this is the second fun-project I do using it, but since I am an **Angular die-hard** I have to try the stable version of [Angular 2](https://angular.io/) (which just released RC) to decide **if** there is a winner between them.

If you have a better pipeline for the above tools please contact me :)

[Source code is hosted at Github](https://github.com/lambrospetrou/numerology/)

I decided to play around with <a href="https://www.typescriptlang.org/">Typescript</a> instead of <a href="https://babeljs.io/">BabelJS</a>, which I already tried in the past, for this weekend’s project. 
<h2 id="reasons-for-typescript">Reasons for Typescript</h2>
During the last year I explored several Javascript languages/flavors (transpilers, etc.). If you read my previous articles you should know that I also like <a href="https://www.dartlang.org/">Dart</a>. I started using it in 2014 along with <a href="https://github.com/angular/angular.dart">AngularDart</a> throughout its journey to the stable version 1.0, always going through refactorings and fixing breaking changes. But Dart made me feel again productive and gave me a sensible platform to do front-end development. Unfortunately, it did not pick up as many fans as Google (and me) wanted so it was kinda left behind.
I really believe that the future lies in languages that compile to Javascript or even something else, WebAssembly maybe? (<a href="https://webassembly.github.io/">WebAssembly homepage</a> and <a href="https://medium.com/javascript-scene/what-is-webassembly-the-dawn-of-a-new-era-61256ec5a8f6#.pb2824qir">Article</a> )
I already gave a go to Babel, so this time it was time for Typescript. Why choose Typescript and not stay with Babel is justified by the points below.
<ul>
<li>I love static typed languages :) and Typescript is much closer to that rather than JS</li>
<li>I prefer compilation errors much more than runtime exceptions</li>
<li>I believe it will be alive for at least 3 more years since it is backed by Microsoft and Google and is used as the main language for Angular 2</li>
<li>Better IDE support (if I decide to use one)</li>
</ul>
<h2 id="project">Project</h2>
I wanted to not spend more than 1-2 days so I have chosen something super simple to implement. I decided to develop a numerology calculator of your life path number :) Fancy stuff!
<a href="/articles/webdev-typescript-webpack-react-numerology/demo/">Calculate your life path number at the demo page</a>
<h2 id="build-process">Build Process</h2>
My main goal with the project was to come up with a build process using Typescript with ES6, Webpack and ReactJS. I also wanted to use Autoprefixer and SASS so I added Gulp to the mix too, but I tried to keep it at a bare minimum.
The final process is explained in the following sections.
<h3 id="code">Code</h3>
I wanted from the beginning to consolidate the code building into one tool, and the decision was pretty easy since <a href="http://webpack.github.io/">Webpack</a> is the big winner (at least for now).
The requirements are simple:
<ul>
<li>Write my code in Typescript</li>
<li>Use ES6 features</li>
<li>Support browsers that only have ES5 support</li>
<li>As a bonus I also wrote the gulpfile in Typescript and used a hack-around to use it to actually compile the Typescript code :)</li>
</ul>
For ReactJS I needed to install react and react-dom along with their typings to allow Typescript to resolve the React types. Pretty smooth process and I also found out something I did not know before. With webpack I can use the npm modules during development but do not bundle them along with my app code to keep the footprint small. By specifying that these modules are external I just need to use script tags in my HTML to import the public ReactJS libraries and take advantage of CDNs and local cached versions.
<pre><code class="language-bash">npm install typescript typings react-dom react --save-dev
typings install dt~react dt~react-dom --global --save</code></pre>
In order to be able to use some of the ES6 features, like Object.assign(), I had to include <a href="https://cdnjs.com/libraries/es5-shim">es5-shim</a> and <a href="https://cdnjs.com/libraries/es6-shim">es6-shim</a> in my HTML code right before I imported the Typescript compiled bundled which targets ES5. I also needed to install the typings for ES6 in order to stop Typescript compilation errors when using ES6 features in the code.
<pre><code class="language-bash">typings install dt~es6-shim --global --save</code></pre>
So, the Typescript compiler will use the typings and compile the code into ES5-valid Javascript and then Webpack will take all the files and output 2 bundles. I like to have separation between the app code which is Typescript and other JS code I might have, so I compile a bundle for the Typescript code ts.bundle.js and one for normal JS-code js.bundle.js.
Use Typescript for the Gulpfile
I could have used regular ES5 Javascript to write the Gulpfile but I wanted to play more with Typescript. I spend some time to be honest searching for solutions for this and I ended up following <a href="https://medium.com/@pleerock/create-a-gulpfile-and-write-gulp-tasks-using-typescript-f08edebcac57#.55q6zomio">this awesome guide</a> adapting it where needed.
<h3 id="sass-and-build">SASS and Build</h3>
No surprises here! Gulp is used to clean the build directory, to use <a href="https://github.com/postcss/autoprefixer">Autoprefixer</a> to prefix new/useful CSS with browser-prefixes and to build the final CSS using <a href="http://sass-lang.com/">SASS</a>.
<h2 id="conclusion">Conclusion</h2>
I spend some time back and forth with the Typescript documentation, expected since this is the first time I tried it, but I liked it. I am going to stick with this for a while until I find something much more productive and powerful, <a href="https://www.scala-js.org/">ScalaJS</a> might be that one :)
Finally, I liked ReactJS too, and this is the second fun-project I do using it, but since I am an Angular die-hard I have to try the stable version of <a href="https://angular.io/">Angular 2</a> (which just released RC) to decide if there is a winner between them.
If you have a better pipeline for the above tools please contact me :)
<a href="https://github.com/lambrospetrou/numerology/">Source code is hosted at Github</a>

A small fun-project with Typescript, Webpack and ReactJS. Numerology calculates your life path number.

Fun with Typescript, Webpack and ReactJS

I wanted to write this article for a while now but I waited until I actually passed the certification exam before I started giving tips to others :) 

This article is mostly a reference for myself, but it might also be helpful to anyone preparing for the [AWS Certifications](https://aws.amazon.com/certification/). I used the following resource material myself over the last couple of months and since I passed the exam with a pretty high mark I can assume they are good enough to recommend them :)

First of all, you have to bookmark the official [AWS Certification Preparation Guide](http://aws.amazon.com/certification/certification-prep/).

## AWS videos <a name="aws-videos"></a>

The first thing I started doing when preparing for the exams, [Solutions Architect](http://aws.amazon.com/certification/certified-solutions-architect-associate/) and [Developer](http://aws.amazon.com/certification/certified-developer-associate/), is watching all the re:Invent videos from 2015.

It might seem that they are just too many to watch or that you will spend huge amount of time, but trust me, they are going to help you more than any documentation or tips or online trainings. They will not only give you insights for individual services but also architectural design best-practices which you can apply into your own work environment even if you are not AWS-hosted.

Personally, I watched pretty much all the 2015 re:Invent videos over the span of 3 months, but you can just focus on the **fundamentals** and the **deep dives** and skip the **war stories** from several companies.

Below you will find the links to the **AWS re:Invent 2015** playlist on Youtube and to the **AWS Summit 2016 - Chicago** playlist which is just a few weeks old :)

The re:Invent videos are marked with a department and a 3-digit code denoting the pre-requisite knowledge you need to have in order to being able to follow the speaker. Remember the college course naming scheme? It's the same!

### AWS re:Invent 2015 <a name="aws-reinvent-2016-videos"></a>

You can find the videos at [AWS re:Invent 2015 Youtube playlist](https://www.youtube.com/user/AmazonWebServices/playlists?view=50&shelf_id=15&sort=dd)

Some of my favourites are (no particular order):

* [(SEC302) IAM Best Practices to Live By](https://www.youtube.com/watch?v=_wiGpBQGCjU&list=PLhr1KZpdzukc9aw8-gnLmyralfsBv7zcR&index=11)
* [(SEC305) How to Become an IAM Policy Ninja in 60 Minutes or Less](https://www.youtube.com/watch?v=Du478i9O_mc&list=PLhr1KZpdzukc9aw8-gnLmyralfsBv7zcR&index=13)
* [(STG201) State of the Union: AWS Storage Services](https://www.youtube.com/watch?v=3HDQsW_r1DM&list=PLhr1KZpdzukdTMmq1gkXs7g6WIIXtL5r9&index=1)
* [(STG401) Amazon S3 Deep Dive and Best Practices](https://www.youtube.com/watch?v=1TvJCLl9NNg&list=PLhr1KZpdzukdTMmq1gkXs7g6WIIXtL5r9&index=8)
* [(STG206) Using Amazon CloudFront For Your Websites & Apps](https://www.youtube.com/watch?v=gUAuhdtHacI&list=PLhr1KZpdzukdTMmq1gkXs7g6WIIXtL5r9&index=13)
* [(ARC307) Infrastructure as Code](https://www.youtube.com/watch?v=WL2xSMVXy5w&index=10&list=PLhr1KZpdzukdRxs_pGJm-qSy5LayL6W_Y)
* [(CMP201) All You Need To Know About Auto Scaling](https://www.youtube.com/watch?v=4trGuelatMI&list=PLhr1KZpdzukfVW6NrpDzdT6Sej0p5POkN&index=6)
* [(CMP402) Amazon EC2 Instances Deep Dive](https://www.youtube.com/watch?v=SZAvtbrIBAk&list=PLhr1KZpdzukfVW6NrpDzdT6Sej0p5POkN&index=11)
* [(DAT407) Amazon ElastiCache: Deep Dive](https://www.youtube.com/watch?v=4VfIINg9DYI&index=6&list=PLhr1KZpdzukeMbjRqGswHX38DCqOHZ5GA)
* [(DAT401) Amazon DynamoDB Deep Dive](https://www.youtube.com/watch?v=ggDIat_FZtA&index=16&list=PLhr1KZpdzukeMbjRqGswHX38DCqOHZ5GA)
* [(NET201) VPC Fundamentals and Connectivity Options](https://www.youtube.com/watch?v=5_bQ6Dgk6k8&index=1&list=PLhr1KZpdzukcjwZgFBBTmSNPjf_gImgfx)
* [(NET406) Deep Dive: AWS Direct Connect and VPNs](https://www.youtube.com/watch?v=SMvom9QjkPk&index=2&list=PLhr1KZpdzukcjwZgFBBTmSNPjf_gImgfx)

The above are just a few selections to **start with** and by no means conclusive. I also recommend you watching **re:Invent 2014 and 2013** videos especially **deep-dives**.

### AWS Summit 2016 - Chicago <a name="aws-summit-2016-chicago-videos"></a>

You can find the videos at [AWS Summit 2016 - Chicago]( https://www.youtube.com/playlist?list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez)

This videos are not as structured as the re:Invent ones but you can as easily filter out the service-oriented ones from the war-story-oriented ones.

Again a few selections:

* [Deep Dive on Amazon Relational Database Service](https://www.youtube.com/watch?v=9-7azhB27So&list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez&index=23)
* [DevOps on AWS](https://www.youtube.com/watch?v=652Wf1KKedk&list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez&index=17)
* [Deep Dive on Amazon Elastic Block Store](https://www.youtube.com/watch?v=MDeKncXDAgk&list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez&index=41)

There is some overlap with re:Invent videos in most of the summit ones but there is also additional valuable content.

## AWS Whitepapers <a name="aws-whitepapers"></a>

**Whitepapers** are my second favourite resources after the re:Invent videos. They present some of the most important topics of deploying services on AWS and provide best-practices and pitfalls to avoid in the process.

You can find them all at [AWS Whitepapers](http://aws.amazon.com/whitepapers/).

Recommended reading:

* [Building Static Websites on AWS - an Astonishing Modern Architecture](http://d0.awsstatic.com/whitepapers/Building%20Static%20Websites%20on%20AWS.pdf)
* [AWS Cloud Storage Services Overview](http://d0.awsstatic.com/whitepapers/AWS%20Storage%20Services%20Whitepaper-v9.pdf)
* [AWS Well-Architected Framework](http://d0.awsstatic.com/whitepapers/architecture/AWS_Well-Architected_Framework.pdf)
* [Architecting for the AWS Cloud: Best Practices](http://d0.awsstatic.com/whitepapers/AWS_Cloud_Best_Practices.pdf)
* [Introduction to AWS Security](http://d0.awsstatic.com/whitepapers/Security/Intro_to_AWS_Security.pdf)
* [AWS Security Best Practices](http://d0.awsstatic.com/whitepapers/aws-security-best-practices.pdf)
* [Overview of Deployment Options on AWS](http://d0.awsstatic.com/whitepapers/overview-of-deployment-options-on-aws.pdf)
* [AWS Securing Data at Rest with Encryption](https://d0.awsstatic.com/whitepapers/AWS_Securing_Data_at_Rest_with_Encryption.pdf)

## Documentation and FAQs <a name="doc-faq"></a>

Another great resource is obviously the documentation of each service and its FAQ. I would strongly advise you to go over the **FAQ** of the most important services, depending on your exam, and if you have time then go over the docs. 

I have to warn you :) **FAQs** are a **must!!!**

You can find links to most of the FAQs at https://aws.amazon.com/faqs/
Or you can visit the FAQ of the service you want directly by visiting the appropriate link, **aws.amazon.com/SERVICENAME/faqs/**. For example, the Simple Storage Service (S3) FAQ will be at **aws.amazon.com/s3/faqs/**.

Personally, I read the FAQs while travelling in the aeroplane or in the tube/buses. As a result, I created a small script to extract the main content out of the website of each FAQ and I put them on my Kindle :) resulting in a much better and easier read!

You can find the small script on Github: [AWS FAQ client](https://github.com/lambrospetrou/aws-faq-client/)

## AWS Training classes <a name="aws-training-class"></a>

The official [AWS Certification Preparation Guide](http://aws.amazon.com/certification/certification-prep/) suggests to attend training classes depending on the exam you want to pass.

I attended two of them so far, the [Developing on AWS](http://aws.amazon.com/training/course-descriptions/developing/) and the [System Operations on AWS](http://aws.amazon.com/training/course-descriptions/sysops/). I have to admit that the developing one apart from some advices and tips from the trainer himself it was very basic and I would not recommend it to someone that already uses AWS. However, if you are just starting with AWS it is a **great** introduction to the **core** services. The **SysOps** training on the other hand was **amazing**. I highly recommend this training to anyone because it covers things from advanced monitoring, to tagging, to cost-optimization, to custom AMIs and deployment options and pretty much things that I doubt you will ever do on your own or if you are not already working as a systems administrator.

**Update @2016-05-30**

Last week, I attended the [Architecting on AWS](http://aws.amazon.com/training/course-descriptions/architect/) training course. I would really recommend it to anyone that will attempt the certification exam since it is a very nice overview of all the architectural aspects of AWS. It goes over specific use-cases, optimizes and finds bottlenecks of the architecture for certain scenarios which requires you to know about several services and how they can be used together.

Overall, if money is no problem (most companies cover the expenses anyway), I would really recommend you to attend these 3-day training courses. But, it really depends on your knowledge and expertise level. 

## AWS Qwiklabs <a name="aws-qwiklabs"></a>

I highly recommend you to do as many [AWS Qwiklabs](https://run.qwiklab.com/) as you can. They range from quick to long practical labs on most of the services and basic concepts you will need for the exams and for actual work on AWS.

Start by doing the **free** ones first, but I really encourage you to try some of the advanced ones too. Even better if you could convince your company to pay for them. Not only they cover in-depth concepts but they allow you to use expensive services that would be cost-prohibitive to use and play with using your own personal account. 

## Projects, projects, projects <a name="projects"></a>

**Most Important**

As with anything you want to learn, actual practice provides the best learning experience.

I did several small projects, and wrote about them in previous articles, where I actually used most of the services covered by the exams. This provided me with invaluable knowledge and insights as to how some services work which I could not acquire just by reading stuff. After all, the exam is called **Developer Associate** because you need to be able to develop and build systems, not just know about them :) I can assure you that the exam contains questions that you cannot answer correctly just by reading the docs.

**Qwiklabs** can significantly contribute to this if you opt-in for the advanced labs but again, from personal experience, if you do not build something from scratch on your own you do not really learn.

## Conclusion

I provided the most important resources I used to prepare for the exam and get myself acquainted with most of the AWS services. I am pretty sure if you follow my advices you will have great results, but by no means consider this an exhaustive list of content or tips.

Watch as many videos as you can from all the **re:Invents**, do as many **qwiklabs** as you can, and most importantly try to do small (or big) projects utilising a variety of AWS services.

The content of the different examinations has a significant amount of overlap, **but** focuses on different aspects of each topic. For example, the **developer** exam will concentrate on performance, API, and implementation details of a service whereas the **architect** exam will focus on orchestrating and utilising the correct services to solve a problem.

**Good luck :)**

I wanted to write this article for a while now but I waited until I actually passed the certification exam before I started giving tips to others :) 
This article is mostly a reference for myself, but it might also be helpful to anyone preparing for the <a href="https://aws.amazon.com/certification/">AWS Certifications</a>. I used the following resource material myself over the last couple of months and since I passed the exam with a pretty high mark I can assume they are good enough to recommend them :)
First of all, you have to bookmark the official <a href="http://aws.amazon.com/certification/certification-prep/">AWS Certification Preparation Guide</a>.
<h2 id="aws-videos-">AWS videos <a name="aws-videos"></a></h2>
The first thing I started doing when preparing for the exams, <a href="http://aws.amazon.com/certification/certified-solutions-architect-associate/">Solutions Architect</a> and <a href="http://aws.amazon.com/certification/certified-developer-associate/">Developer</a>, is watching all the re:Invent videos from 2015.
It might seem that they are just too many to watch or that you will spend huge amount of time, but trust me, they are going to help you more than any documentation or tips or online trainings. They will not only give you insights for individual services but also architectural design best-practices which you can apply into your own work environment even if you are not AWS-hosted.
Personally, I watched pretty much all the 2015 re:Invent videos over the span of 3 months, but you can just focus on the fundamentals and the deep dives and skip the war stories from several companies.
Below you will find the links to the AWS re:Invent 2015 playlist on Youtube and to the AWS Summit 2016 - Chicago playlist which is just a few weeks old :)
The re:Invent videos are marked with a department and a 3-digit code denoting the pre-requisite knowledge you need to have in order to being able to follow the speaker. Remember the college course naming scheme? It’s the same!
<h3 id="aws-reinvent-2015-">AWS re:Invent 2015 <a name="aws-reinvent-2016-videos"></a></h3>
You can find the videos at <a href="https://www.youtube.com/user/AmazonWebServices/playlists?view=50&amp;shelf_id=15&amp;sort=dd">AWS re:Invent 2015 Youtube playlist</a>
Some of my favourites are (no particular order):
<ul>
<li><a href="https://www.youtube.com/watch?v=_wiGpBQGCjU&amp;list=PLhr1KZpdzukc9aw8-gnLmyralfsBv7zcR&amp;index=11">(SEC302) IAM Best Practices to Live By</a></li>
<li><a href="https://www.youtube.com/watch?v=Du478i9O_mc&amp;list=PLhr1KZpdzukc9aw8-gnLmyralfsBv7zcR&amp;index=13">(SEC305) How to Become an IAM Policy Ninja in 60 Minutes or Less</a></li>
<li><a href="https://www.youtube.com/watch?v=3HDQsW_r1DM&amp;list=PLhr1KZpdzukdTMmq1gkXs7g6WIIXtL5r9&amp;index=1">(STG201) State of the Union: AWS Storage Services</a></li>
<li><a href="https://www.youtube.com/watch?v=1TvJCLl9NNg&amp;list=PLhr1KZpdzukdTMmq1gkXs7g6WIIXtL5r9&amp;index=8">(STG401) Amazon S3 Deep Dive and Best Practices</a></li>
<li><a href="https://www.youtube.com/watch?v=gUAuhdtHacI&amp;list=PLhr1KZpdzukdTMmq1gkXs7g6WIIXtL5r9&amp;index=13">(STG206) Using Amazon CloudFront For Your Websites &amp; Apps</a></li>
<li><a href="https://www.youtube.com/watch?v=WL2xSMVXy5w&amp;index=10&amp;list=PLhr1KZpdzukdRxs_pGJm-qSy5LayL6W_Y">(ARC307) Infrastructure as Code</a></li>
<li><a href="https://www.youtube.com/watch?v=4trGuelatMI&amp;list=PLhr1KZpdzukfVW6NrpDzdT6Sej0p5POkN&amp;index=6">(CMP201) All You Need To Know About Auto Scaling</a></li>
<li><a href="https://www.youtube.com/watch?v=SZAvtbrIBAk&amp;list=PLhr1KZpdzukfVW6NrpDzdT6Sej0p5POkN&amp;index=11">(CMP402) Amazon EC2 Instances Deep Dive</a></li>
<li><a href="https://www.youtube.com/watch?v=4VfIINg9DYI&amp;index=6&amp;list=PLhr1KZpdzukeMbjRqGswHX38DCqOHZ5GA">(DAT407) Amazon ElastiCache: Deep Dive</a></li>
<li><a href="https://www.youtube.com/watch?v=ggDIat_FZtA&amp;index=16&amp;list=PLhr1KZpdzukeMbjRqGswHX38DCqOHZ5GA">(DAT401) Amazon DynamoDB Deep Dive</a></li>
<li><a href="https://www.youtube.com/watch?v=5_bQ6Dgk6k8&amp;index=1&amp;list=PLhr1KZpdzukcjwZgFBBTmSNPjf_gImgfx">(NET201) VPC Fundamentals and Connectivity Options</a></li>
<li><a href="https://www.youtube.com/watch?v=SMvom9QjkPk&amp;index=2&amp;list=PLhr1KZpdzukcjwZgFBBTmSNPjf_gImgfx">(NET406) Deep Dive: AWS Direct Connect and VPNs</a></li>
</ul>
The above are just a few selections to start with and by no means conclusive. I also recommend you watching re:Invent 2014 and 2013 videos especially deep-dives.
<h3 id="aws-summit-2016---chicago-">AWS Summit 2016 - Chicago <a name="aws-summit-2016-chicago-videos"></a></h3>
You can find the videos at <a href="https://www.youtube.com/playlist?list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez">AWS Summit 2016 - Chicago</a>
This videos are not as structured as the re:Invent ones but you can as easily filter out the service-oriented ones from the war-story-oriented ones.
Again a few selections:
<ul>
<li><a href="https://www.youtube.com/watch?v=9-7azhB27So&amp;list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez&amp;index=23">Deep Dive on Amazon Relational Database Service</a></li>
<li><a href="https://www.youtube.com/watch?v=652Wf1KKedk&amp;list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez&amp;index=17">DevOps on AWS</a></li>
<li><a href="https://www.youtube.com/watch?v=MDeKncXDAgk&amp;list=PLhr1KZpdzukc2_5o7YTT7e2dlKBEKR1ez&amp;index=41">Deep Dive on Amazon Elastic Block Store</a></li>
</ul>
There is some overlap with re:Invent videos in most of the summit ones but there is also additional valuable content.
<h2 id="aws-whitepapers-">AWS Whitepapers <a name="aws-whitepapers"></a></h2>
Whitepapers are my second favourite resources after the re:Invent videos. They present some of the most important topics of deploying services on AWS and provide best-practices and pitfalls to avoid in the process.
You can find them all at <a href="http://aws.amazon.com/whitepapers/">AWS Whitepapers</a>.
Recommended reading:
<ul>
<li><a href="http://d0.awsstatic.com/whitepapers/Building%20Static%20Websites%20on%20AWS.pdf">Building Static Websites on AWS - an Astonishing Modern Architecture</a></li>
<li><a href="http://d0.awsstatic.com/whitepapers/AWS%20Storage%20Services%20Whitepaper-v9.pdf">AWS Cloud Storage Services Overview</a></li>
<li><a href="http://d0.awsstatic.com/whitepapers/architecture/AWS_Well-Architected_Framework.pdf">AWS Well-Architected Framework</a></li>
<li><a href="http://d0.awsstatic.com/whitepapers/AWS_Cloud_Best_Practices.pdf">Architecting for the AWS Cloud: Best Practices</a></li>
<li><a href="http://d0.awsstatic.com/whitepapers/Security/Intro_to_AWS_Security.pdf">Introduction to AWS Security</a></li>
<li><a href="http://d0.awsstatic.com/whitepapers/aws-security-best-practices.pdf">AWS Security Best Practices</a></li>
<li><a href="http://d0.awsstatic.com/whitepapers/overview-of-deployment-options-on-aws.pdf">Overview of Deployment Options on AWS</a></li>
<li><a href="https://d0.awsstatic.com/whitepapers/AWS_Securing_Data_at_Rest_with_Encryption.pdf">AWS Securing Data at Rest with Encryption</a></li>
</ul>
<h2 id="documentation-and-faqs-">Documentation and FAQs <a name="doc-faq"></a></h2>
Another great resource is obviously the documentation of each service and its FAQ. I would strongly advise you to go over the FAQ of the most important services, depending on your exam, and if you have time then go over the docs. 
I have to warn you :) FAQs are a must!!!
You can find links to most of the FAQs at <a href="https://aws.amazon.com/faqs/">https://aws.amazon.com/faqs/</a>
Or you can visit the FAQ of the service you want directly by visiting the appropriate link, aws.amazon.com/SERVICENAME/faqs/. For example, the Simple Storage Service (S3) FAQ will be at aws.amazon.com/s3/faqs/.
Personally, I read the FAQs while travelling in the aeroplane or in the tube/buses. As a result, I created a small script to extract the main content out of the website of each FAQ and I put them on my Kindle :) resulting in a much better and easier read!
You can find the small script on Github: <a href="https://github.com/lambrospetrou/aws-faq-client/">AWS FAQ client</a>
<h2 id="aws-training-classes-">AWS Training classes <a name="aws-training-class"></a></h2>
The official <a href="http://aws.amazon.com/certification/certification-prep/">AWS Certification Preparation Guide</a> suggests to attend training classes depending on the exam you want to pass.
I attended two of them so far, the <a href="http://aws.amazon.com/training/course-descriptions/developing/">Developing on AWS</a> and the <a href="http://aws.amazon.com/training/course-descriptions/sysops/">System Operations on AWS</a>. I have to admit that the developing one apart from some advices and tips from the trainer himself it was very basic and I would not recommend it to someone that already uses AWS. However, if you are just starting with AWS it is a great introduction to the core services. The SysOps training on the other hand was amazing. I highly recommend this training to anyone because it covers things from advanced monitoring, to tagging, to cost-optimization, to custom AMIs and deployment options and pretty much things that I doubt you will ever do on your own or if you are not already working as a systems administrator.
Update @2016-05-30
Last week, I attended the <a href="http://aws.amazon.com/training/course-descriptions/architect/">Architecting on AWS</a> training course. I would really recommend it to anyone that will attempt the certification exam since it is a very nice overview of all the architectural aspects of AWS. It goes over specific use-cases, optimizes and finds bottlenecks of the architecture for certain scenarios which requires you to know about several services and how they can be used together.
Overall, if money is no problem (most companies cover the expenses anyway), I would really recommend you to attend these 3-day training courses. But, it really depends on your knowledge and expertise level. 
<h2 id="aws-qwiklabs-">AWS Qwiklabs <a name="aws-qwiklabs"></a></h2>
I highly recommend you to do as many <a href="https://run.qwiklab.com/">AWS Qwiklabs</a> as you can. They range from quick to long practical labs on most of the services and basic concepts you will need for the exams and for actual work on AWS.
Start by doing the free ones first, but I really encourage you to try some of the advanced ones too. Even better if you could convince your company to pay for them. Not only they cover in-depth concepts but they allow you to use expensive services that would be cost-prohibitive to use and play with using your own personal account. 
<h2 id="projects-projects-projects-">Projects, projects, projects <a name="projects"></a></h2>
Most Important
As with anything you want to learn, actual practice provides the best learning experience.
I did several small projects, and wrote about them in previous articles, where I actually used most of the services covered by the exams. This provided me with invaluable knowledge and insights as to how some services work which I could not acquire just by reading stuff. After all, the exam is called Developer Associate because you need to be able to develop and build systems, not just know about them :) I can assure you that the exam contains questions that you cannot answer correctly just by reading the docs.
Qwiklabs can significantly contribute to this if you opt-in for the advanced labs but again, from personal experience, if you do not build something from scratch on your own you do not really learn.
<h2 id="conclusion">Conclusion</h2>
I provided the most important resources I used to prepare for the exam and get myself acquainted with most of the AWS services. I am pretty sure if you follow my advices you will have great results, but by no means consider this an exhaustive list of content or tips.
Watch as many videos as you can from all the re:Invents, do as many qwiklabs as you can, and most importantly try to do small (or big) projects utilising a variety of AWS services.
The content of the different examinations has a significant amount of overlap, but focuses on different aspects of each topic. For example, the developer exam will concentrate on performance, API, and implementation details of a service whereas the architect exam will focus on orchestrating and utilising the correct services to solve a problem.
Good luck :)

Some tips and helpful links for the AWS Developer and Solutions Architect Associate certifications.

AWS Certified Developer & Solution Architect Associate Certification Tips


Finally, I found some time to re-write my super-duper URL shortener yet once again :)

As part of my preparation for the [AWS Certifications](https://aws.amazon.com/certification/) I plan on taking, I wanted to make an architectural re-design of **Spi.to** completely cloud-based on AWS. Last weekend, I finally convinced myself to stop watching shows on [Amazon Video](https://www.amazon.co.uk/av) and did it.

## Architecture overview

In the following diagram we can see an overview of the application's architecture. It is is one of the simplest applications you can make in a weekend but at the same time it allows you to use many cloud services following best-practices in order to achieve fault-tolerance, high availability and durability, which is similar to what you would do in a real super-scalable service.

![Spito Architecture Overview on AWS](/articles/yet-another-spito-rewrite-aws/spito-architecture.png "Spito Architecture Overview on AWS")

You can see that it is a pretty basic setup but without sacrificing performance, availability or durability!

## Services

The services I use in this application can be identified from the diagram above but read below for a small description as to the **why** use each service.

For more information about **Route 53**, **Cloudfront**, and **S3** regarding hosting a static website on AWS you can [read my previous article](/articles/migrate-to-aws-static-website/).

### Amazon Route 53

[Route 53](https://aws.amazon.com/route53/) is a cloud-based DNS management service which I use to manage my domain **spi.to**. I do not use advanced features of the service for this application but you should check it out because it is simply awesome.

### Amazon Cloudfront

[Cloudfront](https://aws.amazon.com/cloudfront/) is one of those services that once you understand how it works and play a little with it, you just love it. It is Amazon's CDN solution but at the same time can act as a reverse proxy to your backend or as a faster gateway to your services instead of going throughout the public network.

I use it in front of my static website server (explained in S3 section) and my REST API service (explained in Elastic Beanstalk section) which are the two origins servicing my application. 

In cloudfront I specify certain behaviors for caching depending on the files but I also include some path patterns to direct each request to the appropriate backend (S3 or API). You can see below a snapshot of the rules I have at the moment.

![Cloudfront behavior rules](/articles/yet-another-spito-rewrite-aws/cloudfront-behavior-rules.png "Cloudfront behavior rules")

The above rules are evaluated top-down until the path matches one rule and that rule is applied without going further down the rule-chain.

You can observe that the first rule ensures that all ```/api/``` calls are going to our **API backend** whilst whatever request comes with ```.``` (dots) or ```/``` (slashes) will go to the **website client** backend. The fourth rule ensures that whatever request comes with **at least** one character, I want to match the **hash ids** now, will go to the API backend. The last rule ensures that the root path ```spi.to/``` without any path (since we covered all other cases) will go to the **website client**, which is the homepage.

There might be a simpler solution, but I could not find a way to use classes of characters in path patterns. If you find one or I missed something in the documentation please contact me :)

Also, another important thing with Cloudfront that a **lot** of people ignore is that you **CAN** use it with dynamic services. In my case the API is strictly dynamic since the spits have expiration dates and they cannot be just cached. You can specify that a path will have no caching, which means that you will just use Cloudfront as a proxy to your service, and as I said before it might be beneficial to your users because the communication to your servers will be done inside the AWS network which has all sorts of optimizations.

### Amazon S3 - Simple Storage Service

I use [Amazon S3](https://aws.amazon.com/s3/) in order to serve the web client (website) which you can access by visiting [http://Spi.to](http://spi.to). The client is a static web application written in Dart which consumes the **Spito API** (REST API explained below).

Amazon S3 as discussed in my previous article, [Hosting a static website at AWS](https://lambrospetrou.com/articles/migrate-to-aws-static-website/), is just amazing for static content (especially when combined with Amazon Cloudfront).

### Amazon DynamoDB

[Amazon DynamoDB](https://aws.amazon.com/dynamodb/) is used to store the text or the URL that you upload to the service. I was considering to use S3 again for that but Dynamo is much faster for small texts. I will update the API in the future and use S3 for large-sized text (now the service only allows you to post text up to 128KB).

DynamoDB has very low latency and allows the service to provide super-fast retrieval of the so-called **Spits** (my naming for the uploaded text) and also provides durability and high-availability of the data.

### AWS Elastic Beanstalk

[Elastic Beanstalk](https://aws.amazon.com/elasticbeanstalk/) is a tool that I just learnt recently, and I loved it instantly. Super-easy to use and you get all the benefits of autoscaling and custom bootstrapping out-of-the-box. You, as a user, just need to upload your source code or binary of the application.

Beanstalk handles the **Spito API** servers inside a managed auto-scaling group which by its turn is behind a managed elastic load balancer, all handled by beanstalk itself.

For this service I use **T2.nano** instances in order to keep the costs at a minimal, in case you were interested :)

The Spito API is written in [#Go](https://golang.org/) (source code found in _Conclusion_ section).

### AWS CloudFormation

[AWS Cloud Formation](https://aws.amazon.com/cloudformation/) is a **future** feature I will add to the project in order to automate the infrastructure creation. As developers we love code, so everything needs to be in code :)

_Coming soon_

## Conclusion

Through this project I learnt a ton about **Elastic Beanstalk** and how to use **Cloudfront** even for dynamic requests, and also re-used the knowledge and tricks from my last project (the aforementioned article about static websites).

I open-sourced the application, both the website and the REST API, but keep in mind that this is **only** the application. I still haven't added the CloudFormation template which creates and bootstraps the required AWS services in an automated way.

Source code

* [Spitoweb client](https://github.com/lambrospetrou/spitoweb)
* [Spito API](https://github.com/lambrospetrou/spito)

_**Warning**: The client (spitoweb) has been written a long time ago so it clearly does not follow Dart best-practices, however it could be useful to read if you are into Dart too :)_

**Ah,** and before I forget, you can find **Spito** at [http://spi.to](http://spi.to).

Any comment or feedback is appreciated. 

## References

* [Migrate to AWS - Make a static website using S3, Cloudfront and Route 53](https://lambrospetrou.com/articles/migrate-to-aws-static-website/)
* [Deploying Applications on the Go Platform](http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-environment.html)
* [Configure the EB CLI](http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-configuration.html)
* [Values that You Specify When You Create or Update a Web Distribution - Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-values-specify.html#DownloadDistValuesPathPattern)
* [Whitepaper - Hosting Static Websites on AWS](https://d0.awsstatic.com/whitepapers/Building%20Static%20Websites%20on%20AWS.pdf)
* [Whitepaper - Overview of Deployment Options on AWS](https://d0.awsstatic.com/whitepapers/overview-of-deployment-options-on-aws.pdf)


Finally, I found some time to re-write my super-duper URL shortener yet once again :)
As part of my preparation for the <a href="https://aws.amazon.com/certification/">AWS Certifications</a> I plan on taking, I wanted to make an architectural re-design of Spi.to completely cloud-based on AWS. Last weekend, I finally convinced myself to stop watching shows on <a href="https://www.amazon.co.uk/av">Amazon Video</a> and did it.
<h2 id="architecture-overview">Architecture overview</h2>
In the following diagram we can see an overview of the application’s architecture. It is is one of the simplest applications you can make in a weekend but at the same time it allows you to use many cloud services following best-practices in order to achieve fault-tolerance, high availability and durability, which is similar to what you would do in a real super-scalable service.
<img src="/articles/yet-another-spito-rewrite-aws/spito-architecture.png" alt="Spito Architecture Overview on AWS" title="Spito Architecture Overview on AWS"/>
You can see that it is a pretty basic setup but without sacrificing performance, availability or durability!
<h2 id="services">Services</h2>
The services I use in this application can be identified from the diagram above but read below for a small description as to the why use each service.
For more information about Route 53, Cloudfront, and S3 regarding hosting a static website on AWS you can <a href="/articles/migrate-to-aws-static-website/">read my previous article</a>.
<h3 id="amazon-route-53">Amazon Route 53</h3>
<a href="https://aws.amazon.com/route53/">Route 53</a> is a cloud-based DNS management service which I use to manage my domain spi.to. I do not use advanced features of the service for this application but you should check it out because it is simply awesome.
<h3 id="amazon-cloudfront">Amazon Cloudfront</h3>
<a href="https://aws.amazon.com/cloudfront/">Cloudfront</a> is one of those services that once you understand how it works and play a little with it, you just love it. It is Amazon’s CDN solution but at the same time can act as a reverse proxy to your backend or as a faster gateway to your services instead of going throughout the public network.
I use it in front of my static website server (explained in S3 section) and my REST API service (explained in Elastic Beanstalk section) which are the two origins servicing my application. 
In cloudfront I specify certain behaviors for caching depending on the files but I also include some path patterns to direct each request to the appropriate backend (S3 or API). You can see below a snapshot of the rules I have at the moment.
<img src="/articles/yet-another-spito-rewrite-aws/cloudfront-behavior-rules.png" alt="Cloudfront behavior rules" title="Cloudfront behavior rules"/>
The above rules are evaluated top-down until the path matches one rule and that rule is applied without going further down the rule-chain.
You can observe that the first rule ensures that all <code>/api/</code> calls are going to our API backend whilst whatever request comes with <code>.</code> (dots) or <code>/</code> (slashes) will go to the website client backend. The fourth rule ensures that whatever request comes with at least one character, I want to match the hash ids now, will go to the API backend. The last rule ensures that the root path <code>spi.to/</code> without any path (since we covered all other cases) will go to the website client, which is the homepage.
There might be a simpler solution, but I could not find a way to use classes of characters in path patterns. If you find one or I missed something in the documentation please contact me :)
Also, another important thing with Cloudfront that a lot of people ignore is that you CAN use it with dynamic services. In my case the API is strictly dynamic since the spits have expiration dates and they cannot be just cached. You can specify that a path will have no caching, which means that you will just use Cloudfront as a proxy to your service, and as I said before it might be beneficial to your users because the communication to your servers will be done inside the AWS network which has all sorts of optimizations.
<h3 id="amazon-s3---simple-storage-service">Amazon S3 - Simple Storage Service</h3>
I use <a href="https://aws.amazon.com/s3/">Amazon S3</a> in order to serve the web client (website) which you can access by visiting <a href="http://spi.to">http://Spi.to</a>. The client is a static web application written in Dart which consumes the Spito API (REST API explained below).
Amazon S3 as discussed in my previous article, <a href="https://lambrospetrou.com/articles/migrate-to-aws-static-website/">Hosting a static website at AWS</a>, is just amazing for static content (especially when combined with Amazon Cloudfront).
<h3 id="amazon-dynamodb">Amazon DynamoDB</h3>
<a href="https://aws.amazon.com/dynamodb/">Amazon DynamoDB</a> is used to store the text or the URL that you upload to the service. I was considering to use S3 again for that but Dynamo is much faster for small texts. I will update the API in the future and use S3 for large-sized text (now the service only allows you to post text up to 128KB).
DynamoDB has very low latency and allows the service to provide super-fast retrieval of the so-called Spits (my naming for the uploaded text) and also provides durability and high-availability of the data.
<h3 id="aws-elastic-beanstalk">AWS Elastic Beanstalk</h3>
<a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a> is a tool that I just learnt recently, and I loved it instantly. Super-easy to use and you get all the benefits of autoscaling and custom bootstrapping out-of-the-box. You, as a user, just need to upload your source code or binary of the application.
Beanstalk handles the Spito API servers inside a managed auto-scaling group which by its turn is behind a managed elastic load balancer, all handled by beanstalk itself.
For this service I use T2.nano instances in order to keep the costs at a minimal, in case you were interested :)
The Spito API is written in <a href="https://golang.org/">#Go</a> (source code found in Conclusion section).
<h3 id="aws-cloudformation">AWS CloudFormation</h3>
<a href="https://aws.amazon.com/cloudformation/">AWS Cloud Formation</a> is a future feature I will add to the project in order to automate the infrastructure creation. As developers we love code, so everything needs to be in code :)
Coming soon
<h2 id="conclusion">Conclusion</h2>
Through this project I learnt a ton about Elastic Beanstalk and how to use Cloudfront even for dynamic requests, and also re-used the knowledge and tricks from my last project (the aforementioned article about static websites).
I open-sourced the application, both the website and the REST API, but keep in mind that this is only the application. I still haven’t added the CloudFormation template which creates and bootstraps the required AWS services in an automated way.
Source code
<ul>
<li><a href="https://github.com/lambrospetrou/spitoweb">Spitoweb client</a></li>
<li><a href="https://github.com/lambrospetrou/spito">Spito API</a></li>
</ul>
Warning: The client (spitoweb) has been written a long time ago so it clearly does not follow Dart best-practices, however it could be useful to read if you are into Dart too :)
Ah, and before I forget, you can find Spito at <a href="http://spi.to">http://spi.to</a>.
Any comment or feedback is appreciated. 
<h2 id="references">References</h2>
<ul>
<li><a href="https://lambrospetrou.com/articles/migrate-to-aws-static-website/">Migrate to AWS - Make a static website using S3, Cloudfront and Route 53</a></li>
<li><a href="http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/go-environment.html">Deploying Applications on the Go Platform</a></li>
<li><a href="http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-configuration.html">Configure the EB CLI</a></li>
<li><a href="http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-values-specify.html#DownloadDistValuesPathPattern">Values that You Specify When You Create or Update a Web Distribution - Cloudfront</a></li>
<li><a href="https://d0.awsstatic.com/whitepapers/Building%20Static%20Websites%20on%20AWS.pdf">Whitepaper - Hosting Static Websites on AWS</a></li>
<li><a href="https://d0.awsstatic.com/whitepapers/overview-of-deployment-options-on-aws.pdf">Whitepaper - Overview of Deployment Options on AWS</a></li>
</ul>

This is an update explaining how I re-wrote Spito to be fully cloud based using AWS. Technologies used include Elastic Beanstalk, Route 53, Cloudfront, S3, and DynamoDB.

Yet another Spito re-write, on the Cloud (AWS)


A few weeks ago a friend asked me for a way to import his Apple Notes into Google Keep. It seems that no easy straighforward way exists as of today so I decided it was a great opportunity to make my first Chrome extension and create my own importer.

It turned out to be a bit more complicated than I expected though, hence no plugin already exists. The reason is that Google as of now does not provide an open API for developers to use in order to import messages which makes the whole project impossible from a programmatible point of view. In the end I managed to overcome this issue and created the extension which you can download from the Chrome store and try it yourself ([direct link to Chrome Store](https://chrome.google.com/webstore/detail/lp-gmail-keep-importer/ingomolknmgnfbafknpkmklapabaednn)).

In this tutorial I will first provide instructions how to use the extension for the impatient ones, and then I will go into more details regarding its implementation and the issues encountered during the process.

## How to use the LP Gmail to Keep Importer

The most updated state and instructions about the plugin will always be accessible at its [Github repository](https://github.com/lambrospetrou/gmail-keep-importer) where you can also download the source code. Any feedback and pull-requests are welcomed.

The following instructions are valid and work with the Google Keep website as of the date of this article.

1. **Install** the plugin from the Chrome store: [Download LP Gmail Keep Importer](https://chrome.google.com/webstore/detail/lp-gmail-keep-importer/ingomolknmgnfbafknpkmklapabaednn)

2. In order for the plugin to work you **have to**, read again, **have to** be at the Google Keep website (reason explained below). Visit [Google Keep](https://keep.google.com/) and login with your account.

3. Once you are at the Keep website, and you have installed the extension you should be able to see the **LP G2K** icon in Chrome's toolbar.

4. **Click** the extension's icon in order to open the plugin and you should see a small window like the picture below.

    ![LP Gmail to Keep Importer - Authorization view](/articles/import-gmail-messages-to-google-keep/authorization.png "LP Gmail to Keep Importer - Authorization")

5. The authorization is required only the first time you use the extension. You have to authorize it to access your emails (**read only** don't worry). 
    * Click **Authorize** to be redirected to Google's authorization page. 
    * Click **Allow** to allow the extension to use your account's email. 
    * The window will stay open with a blank page. This behavior is a bug from Google's side and they are looking into a fix, but for now just close the window yourself and refresh the page.

6. **Click** again the extension's icon and now you should see the normal view with the text box (from now on this is what you should see since you already authorised it).
    
7. In the text box you have to type the **Gmail label name** you want to import. As you know Gmail organizes your mails, messages, in labels so just find out the label you want to import and type it here or create a new one if you don't have one. **Apple Notes** automatically saves your notes as emails under the label name **Notes**, so use that if you want to import your Apple notes.

8. Click **Import GMail Label**. Once the plugin fetches your messages from Gmail, it will show you a small result view with the number of messages found (like the picture below) or it will show an error message if there was a problem during the fetching process.

    ![LP Gmail to Keep Importer - Results view](/articles/import-gmail-messages-to-google-keep/results-view.png "LP Gmail to Keep Importer - Results view")

9. Click **Start Importing messages** in order to start adding your messages as notes into Google Keep. If you have hundreds of messages please **be patient** and wait until you see a response, either an error or a success message.

10. When your messages have been imported you will actually see them in the background to be inside your Keep notes but also you will see a confirmation like the following picture.

    ![LP Gmail to Keep Importer - Confirmation view](/articles/import-gmail-messages-to-google-keep/results-success.png "LP Gmail to Keep Importer - Confirmation view")

11. You have successfully imported your Gmail messages into Google Keep :) 

If you encountered any problems or your experience was different from what I described please contact me in order to fix the issue. As you will read later, the extension is highly experimental and it might not be working as expected in all operating systems (I had some issues with Mac OSX but I tried to resolve them).

In addition, it goes without saying that the importer works for plain emails. If you try to import emails that have attachments or fancy HTML websites embedded only the visible text of the message will be imported. As of version v0.13 the first line of the email becomes the title of the note in Keep.

## Deep dive into the juicy details

This section contains more details about the problem itself, its issues, and the approach that I used to solve it. If you discovered a better solution contact me or even better do a pull-request at Github with your solution and I will adjust it accordingly.

### Problem overview

Initially the problem at hand was to import Apple Notes into Google Keep. Apple Notes stores your notes as messages at Gmail (assuming you use a Google account of course) under the label **Notes** (yes, very innovative name by Apple that will not conflict with anything :D ). This makes it easier for us since the problem now can be reduced to importing from Gmail to Google Keep, which seemed trivial at first glance. Therefore, our problem now is two-fold: **a)** Fetch messages from Gmail and **b)** Create a note in Google Keep with code programmatically. 

Gmail has a very nicely explained and easy to use [Javascript API](https://developers.google.com/gmail/api/quickstart/js) so the first task of fetching the emails under a label name is trivial and I will not go into more details since there are hundreds of articles explaining how to do this. Personally, I find Google's documentation to be superb.

The problem is with the second part, creating notes into Google Keep. Google does **not** provide an open API for Keep as it does with Gmail so far, maybe because it is a new (couple of years) service or just because they don't want developers yet to create apps around it. As a web developer myself I instantly had an idea to overcome this problem. Let's use the Keep website and use the existing form programmatically, thus typing the note, the title and clicking the Done button, all with code. **Nifty idea right? :)**

### The journey inside Google Keep website

My first goal was to come up with a way to identify and reference the three major components we need:

* Title text box
* Content text box
* Done button (more problematic since the actual text might be different in each language)

#### Identification - Selectability

I dived into the HTML source code of the website, always using the awesome Chrome web developer tools, and I tried to detect something distinctive for each one of them. After trying several things the only solution that worked for me since I did not want to use actual text due to translations, is using **CSS classnames**. The only concern I had, have, and will have regarding this is that the classnames are not actual words so I guess Google uses some kind of builder to generate them, so at any moment in the future when Google modifies the Keep website there is a high probability that the classnames will change, thus breaking the plugin. Since I could not find any other identifiable characteristic I decided to go with it and in case this happens in the future I will just update the extension with the new names, until Google release a proper API.

#### Add note with code

So far so good, we have a way to reference each HTML element on the website. Let's try to add some dummy content into the text boxes and simulate the click on the submit button to see what happens. 

    1. Adding text programmatically: Works
    2. Clicking Done: Does not work!

Hmmm, this was not expected at first because the button **was** clicked, but nothing happened and the note disappeared instead of being added. After diving deeper into the code I was examing the HTML dom of the elements in question throughout a proper manual note creation and saw what changed, where the content was added in reality, and what was the button's behavior. I discovered that Google uses two elements per input field instead of one in order to allow expandable text areas. Interesting I thought, so I did a little research and found [this great article by A List Apart](http://alistapart.com/article/expanding-text-areas-made-elegant) which explains in more details how this is done.

**Attempt number two**. Having seen the issue I know referenced the proper element for the input boxes when adding the text, the ones which are really used during the submission of the note. Let's try again.

    1. Adding text programmatically: Works
    2. Clicking Done: Does NOT work!

OK! Google Keep vs Lambros, 2 - 0.

Something was going wrong right? I went over my code several times to make sure I was not being stupid, and strangely enough I wasn't so the problem was somewhere else. I spent several hours investigating the problem and I had an idea. Maybe it was something special happening when the submit button was pressed in reality that I was missing by doing it with code. Again, Chrome developer tools to the rescue.

I discovered a great tool that allows you to inspect the fired HTML/Javascript events of the page and even put breakpoints in your code to the exact point when it happens. 
If you want to see this and test it:

* Open the developer tools in Chrome
* Click the **Sources** tab
* You can see on the right-hand side now a section named **Event Listener Breakpoints**
* Play with it :)

Fast-forward a few hours and I discovered that in order to have a successful note submitted the button when clicked does not only fire the **click** event, but it has to also fire the **mousedown** followed by the **mouseup** events, in this specific order. Additionally, the same has to happen with the input boxes. All the events I use as of now to make a proper submission are shown below.

* Input boxes (title, content)
    1. Add the text to the right element
    2. Fire events: **change** => **mousedown** => **mouseup**
* Submit button
    1. Fire events: **mousedown** => **mouseup**

The implementation source code of this logic and the whole note creation is in the [chrome-logic-keep.js](https://github.com/lambrospetrou/gmail-keep-importer/blob/master/chrome-logic-keep.js) file at Github.

**Attempt number three**. 

    1. Adding text programmatically: Works
    2. Clicking Done: Works

And voila! We can now add notes into Keep programmatically.

After this, I spent several hours figuring out how Chrome extensions work and went ahead and created my first extension. If you encounter any errors, please contact me or contribute to the project on Github. I have already said that it is a **highly experimental** project and it might break at any time if even the smallest detail changes on Google Keep's website.

Google Keep vs Lambros, win for Lambros with KO!

### As Jeff Bezos says, "It's Still Day-One"

It was a very nice little project, and learned tons of new stuff. **Web is amazing and ever-changing** so we have to always keep learning. 


A few weeks ago a friend asked me for a way to import his Apple Notes into Google Keep. It seems that no easy straighforward way exists as of today so I decided it was a great opportunity to make my first Chrome extension and create my own importer.
It turned out to be a bit more complicated than I expected though, hence no plugin already exists. The reason is that Google as of now does not provide an open API for developers to use in order to import messages which makes the whole project impossible from a programmatible point of view. In the end I managed to overcome this issue and created the extension which you can download from the Chrome store and try it yourself (<a href="https://chrome.google.com/webstore/detail/lp-gmail-keep-importer/ingomolknmgnfbafknpkmklapabaednn">direct link to Chrome Store</a>).
In this tutorial I will first provide instructions how to use the extension for the impatient ones, and then I will go into more details regarding its implementation and the issues encountered during the process.
<h2 id="how-to-use-the-lp-gmail-to-keep-importer">How to use the LP Gmail to Keep Importer</h2>
The most updated state and instructions about the plugin will always be accessible at its <a href="https://github.com/lambrospetrou/gmail-keep-importer">Github repository</a> where you can also download the source code. Any feedback and pull-requests are welcomed.
The following instructions are valid and work with the Google Keep website as of the date of this article.
<ol>
<li>Install the plugin from the Chrome store: <a href="https://chrome.google.com/webstore/detail/lp-gmail-keep-importer/ingomolknmgnfbafknpkmklapabaednn">Download LP Gmail Keep Importer</a>
</li>
<li>In order for the plugin to work you have to, read again, have to be at the Google Keep website (reason explained below). Visit <a href="https://keep.google.com/">Google Keep</a> and login with your account.
</li>
<li>Once you are at the Keep website, and you have installed the extension you should be able to see the LP G2K icon in Chrome’s toolbar.
</li>
<li>Click the extension’s icon in order to open the plugin and you should see a small window like the picture below.
 <img src="/articles/import-gmail-messages-to-google-keep/authorization.png" alt="LP Gmail to Keep Importer - Authorization view" title="LP Gmail to Keep Importer - Authorization"/>
</li>
<li>The authorization is required only the first time you use the extension. You have to authorize it to access your emails (read only don’t worry). 
<ul>
<li>Click Authorize to be redirected to Google’s authorization page. </li>
<li>Click Allow to allow the extension to use your account’s email. </li>
<li>The window will stay open with a blank page. This behavior is a bug from Google’s side and they are looking into a fix, but for now just close the window yourself and refresh the page.</li>
</ul>
</li>
<li>Click again the extension’s icon and now you should see the normal view with the text box (from now on this is what you should see since you already authorised it).
</li>
<li>In the text box you have to type the Gmail label name you want to import. As you know Gmail organizes your mails, messages, in labels so just find out the label you want to import and type it here or create a new one if you don’t have one. Apple Notes automatically saves your notes as emails under the label name Notes, so use that if you want to import your Apple notes.
</li>
<li>Click Import GMail Label. Once the plugin fetches your messages from Gmail, it will show you a small result view with the number of messages found (like the picture below) or it will show an error message if there was a problem during the fetching process.
 <img src="/articles/import-gmail-messages-to-google-keep/results-view.png" alt="LP Gmail to Keep Importer - Results view" title="LP Gmail to Keep Importer - Results view"/>
</li>
<li>Click Start Importing messages in order to start adding your messages as notes into Google Keep. If you have hundreds of messages please be patient and wait until you see a response, either an error or a success message.
</li>
<li>When your messages have been imported you will actually see them in the background to be inside your Keep notes but also you will see a confirmation like the following picture.
<img src="/articles/import-gmail-messages-to-google-keep/results-success.png" alt="LP Gmail to Keep Importer - Confirmation view" title="LP Gmail to Keep Importer - Confirmation view"/>
</li>
<li>You have successfully imported your Gmail messages into Google Keep :) 
</li>
</ol>
If you encountered any problems or your experience was different from what I described please contact me in order to fix the issue. As you will read later, the extension is highly experimental and it might not be working as expected in all operating systems (I had some issues with Mac OSX but I tried to resolve them).
In addition, it goes without saying that the importer works for plain emails. If you try to import emails that have attachments or fancy HTML websites embedded only the visible text of the message will be imported. As of version v0.13 the first line of the email becomes the title of the note in Keep.
<h2 id="deep-dive-into-the-juicy-details">Deep dive into the juicy details</h2>
This section contains more details about the problem itself, its issues, and the approach that I used to solve it. If you discovered a better solution contact me or even better do a pull-request at Github with your solution and I will adjust it accordingly.
<h3 id="problem-overview">Problem overview</h3>
Initially the problem at hand was to import Apple Notes into Google Keep. Apple Notes stores your notes as messages at Gmail (assuming you use a Google account of course) under the label Notes (yes, very innovative name by Apple that will not conflict with anything :D ). This makes it easier for us since the problem now can be reduced to importing from Gmail to Google Keep, which seemed trivial at first glance. Therefore, our problem now is two-fold: a) Fetch messages from Gmail and b) Create a note in Google Keep with code programmatically. 
Gmail has a very nicely explained and easy to use <a href="https://developers.google.com/gmail/api/quickstart/js">Javascript API</a> so the first task of fetching the emails under a label name is trivial and I will not go into more details since there are hundreds of articles explaining how to do this. Personally, I find Google’s documentation to be superb.
The problem is with the second part, creating notes into Google Keep. Google does not provide an open API for Keep as it does with Gmail so far, maybe because it is a new (couple of years) service or just because they don’t want developers yet to create apps around it. As a web developer myself I instantly had an idea to overcome this problem. Let’s use the Keep website and use the existing form programmatically, thus typing the note, the title and clicking the Done button, all with code. Nifty idea right? :)
<h3 id="the-journey-inside-google-keep-website">The journey inside Google Keep website</h3>
My first goal was to come up with a way to identify and reference the three major components we need:
<ul>
<li>Title text box</li>
<li>Content text box</li>
<li>Done button (more problematic since the actual text might be different in each language)</li>
</ul>
<h4 id="identification---selectability">Identification - Selectability</h4>
I dived into the HTML source code of the website, always using the awesome Chrome web developer tools, and I tried to detect something distinctive for each one of them. After trying several things the only solution that worked for me since I did not want to use actual text due to translations, is using CSS classnames. The only concern I had, have, and will have regarding this is that the classnames are not actual words so I guess Google uses some kind of builder to generate them, so at any moment in the future when Google modifies the Keep website there is a high probability that the classnames will change, thus breaking the plugin. Since I could not find any other identifiable characteristic I decided to go with it and in case this happens in the future I will just update the extension with the new names, until Google release a proper API.
<h4 id="add-note-with-code">Add note with code</h4>
So far so good, we have a way to reference each HTML element on the website. Let’s try to add some dummy content into the text boxes and simulate the click on the submit button to see what happens. 
<pre><code>1. Adding text programmatically: Works
2. Clicking Done: Does not work!</code></pre>Hmmm, this was not expected at first because the button was clicked, but nothing happened and the note disappeared instead of being added. After diving deeper into the code I was examing the HTML dom of the elements in question throughout a proper manual note creation and saw what changed, where the content was added in reality, and what was the button’s behavior. I discovered that Google uses two elements per input field instead of one in order to allow expandable text areas. Interesting I thought, so I did a little research and found <a href="http://alistapart.com/article/expanding-text-areas-made-elegant">this great article by A List Apart</a> which explains in more details how this is done.
Attempt number two. Having seen the issue I know referenced the proper element for the input boxes when adding the text, the ones which are really used during the submission of the note. Let’s try again.
<pre><code>1. Adding text programmatically: Works
2. Clicking Done: Does NOT work!</code></pre>OK! Google Keep vs Lambros, 2 - 0.
Something was going wrong right? I went over my code several times to make sure I was not being stupid, and strangely enough I wasn’t so the problem was somewhere else. I spent several hours investigating the problem and I had an idea. Maybe it was something special happening when the submit button was pressed in reality that I was missing by doing it with code. Again, Chrome developer tools to the rescue.
I discovered a great tool that allows you to inspect the fired HTML/Javascript events of the page and even put breakpoints in your code to the exact point when it happens. 
If you want to see this and test it:
<ul>
<li>Open the developer tools in Chrome</li>
<li>Click the Sources tab</li>
<li>You can see on the right-hand side now a section named Event Listener Breakpoints</li>
<li>Play with it :)</li>
</ul>
Fast-forward a few hours and I discovered that in order to have a successful note submitted the button when clicked does not only fire the click event, but it has to also fire the mousedown followed by the mouseup events, in this specific order. Additionally, the same has to happen with the input boxes. All the events I use as of now to make a proper submission are shown below.
<ul>
<li>Input boxes (title, content)<ol>
<li>Add the text to the right element</li>
<li>Fire events: change =&gt; mousedown =&gt; mouseup</li>
</ol>
</li>
<li>Submit button<ol>
<li>Fire events: mousedown =&gt; mouseup</li>
</ol>
</li>
</ul>
The implementation source code of this logic and the whole note creation is in the <a href="https://github.com/lambrospetrou/gmail-keep-importer/blob/master/chrome-logic-keep.js">chrome-logic-keep.js</a> file at Github.
Attempt number three. 
<pre><code>1. Adding text programmatically: Works
2. Clicking Done: Works</code></pre>And voila! We can now add notes into Keep programmatically.
After this, I spent several hours figuring out how Chrome extensions work and went ahead and created my first extension. If you encounter any errors, please contact me or contribute to the project on Github. I have already said that it is a highly experimental project and it might break at any time if even the smallest detail changes on Google Keep’s website.
Google Keep vs Lambros, win for Lambros with KO!
<h3 id="as-jeff-bezos-says-its-still-day-one">As Jeff Bezos says, “It’s Still Day-One”</h3>
It was a very nice little project, and learned tons of new stuff. Web is amazing and ever-changing so we have to always keep learning.

This a tutorial for a Chrome extension that allows you to import Gmail messages into Google Keep as notes. It can even be used to import Apple Notes into Keep.

Import Gmail messages into Google Keep as notes

This tutorial explains in detail how to setup a static website with unlimited throughput, durability and availability using Amazon S3. Additionally, we exploit Amazon Cloudfront to provide caching and super-fast downloads all over the world and finally, we use Route 53 to provide the website behind our custom domain name.

Migrate to AWS - Make a static website using S3, Cloudfront and Route 53

A common feature among most websites is a way to contact the owner for any comments or feedback (or complaints). In previous versions of my website and blog I used several techniques to provide a contact me form. In almost all the cases I had to implement the actual email submission on the website's hosting server or in some cases used a 3rd party service that provided me this functionality by querying an HTTP endpoint they gave me with the actual message and they sent the email for me, to me.

Today I decided to roll out my own contact-form service using AWS. As it turns out, it is amazingly easy to setup and it only takes you 15 minutes from the moment you sign up for the free-tier account to the point where you have an active form sending you emails. **Awesome time we live in!**

## Assumptions

- You have an active AWS account (if not do yourself a favor and register for the 1-year free tier at [AWS](https://aws.amazon.com/)).
- You have a website somewhere or you know how to use Simple Storage Service (S3) from AWS to host your website files.

## Services

In order to implement the contact form we are gong to use two services from AWS.

1. **Simple Notification Service (SNS)** ([SNS service](https://aws.amazon.com/sns/))
 Amazon SNS is a fast, flexible, fully managed pub-sub messaging service. Use it as a cloud-based mobile app notification service to send push notifications, email, and SMS messages; or as an enterprise-messaging infrastructure. _(description as per AWS documentation)_

2. **Amazon Cognito** ([Cognito service](https://aws.amazon.com/cognito/))
 Amazon Cognito gives you unique identifiers for your end users and then lets you securely store and sync user app data in the AWS Cloud across multiple devices and OS platforms. You can do this with just a few lines of code, and your app can work the same, regardless of whether a user’s devices are online or offline. When new data is available in the sync store, a user’s devices can be alerted by a silent push notification so that your app can sync the new data automatically. _(description as per AWS documentation)_


## Procedure

The whole procedure is very simple. We will create an SNS topic (imagine it as an endpoint) where the website will send the message of the contact form. As soon as SNS receives the message it will automatically send it over to our email, which we will have configured through the AWS console to be a subscriber of that specific topic. Amazon Cognito is used just as a way to get temporary credentials for the website in order to be able to publish the message to the SNS topic.

I will explain the implementation in three steps, **a)** Amazon Cognito, **b)** SNS and **c)** Securing our SNS topic to allow only publishings from our website domain.


### Amazon Cognito

We use Cognito just for the sake of getting temporary credentials for our website in order to be able to use any AWS service through the Javascript SDK.

1. Login to the AWS console and open the **Amazon Cognito** service ([direct link](https://console.aws.amazon.com/cognito/))

2. Select **Create new identity pool**

 ![Amazon Cognito - Create new identity pool](/articles/create-contact-form-using-aws-sns/data/cognito-create-pool.png "Amazon Cognito - Create new identity pool")

3. Type in a name for your pool (i.e. Website_Contact_Form) and enable **Unauthenticated identities**.

4. You should have something along the lines of the following picture.

 ![Amazon Cognito - Identity pool information](/articles/create-contact-form-using-aws-sns/data/cognito-info-pool.png "Amazon Cognito - Identity pool information")

5. Click **Create Pool** and then on the next page you get the message "Your Cognito identities require access to your resources". Just click **Allow** (bottom right) to proceed.

6. You should be in the **Edit identity pool** page now where you see the information you provided above. From the left side menu choose **Sample code** and in the **Platform** dropdown select Javascript.

7. Download the AWS SDK since you will use it on your website later.

8. You can use the code provided to get temporary credentials for the newly created identity pool. In my case the code is similar to what is shown below.

 ```Javascript
 // Initialize the Amazon Cognito credentials provider
 AWS.config.region = 'eu-west-1'; // Region
 AWS.config.credentials = new AWS.CognitoIdentityCredentials({
 IdentityPoolId: 'eu-west-1:e4c24108-5050-42f8-ac0b-761c46aa947f',
 });
 ```

That's it for the Cognito service!

### Simple Notification Service (SNS)

SNS is a bit longer procedure but nothing complex. The philosophy behind SNS is that you have a topic (endpoint) with some publishers and some subscribers. When a publisher posts something to that topic then all the subscribers are notified automatically by the service with the newly published message. I guess by now you should have figured out that the subscriber to our SNS topic will be our *email* and the publisher of the topic will be our _website_.

Let's dive in!

1. Login to the AWS console and open the **SNS** service ([direct link](https://console.aws.amazon.com/sns/))

 ![SNS - Create Topic](/articles/create-contact-form-using-aws-sns/data/sns-create-topic.png "SNS - Create Topic")

2. Select **Create Topic** and then just fill out the **Topic name** with whatever you want (i.e. com-website-contact-form). You should now see the **Topic Details** page for your new topic (similar to the picture below).

 ![SNS - Topic details](/articles/create-contact-form-using-aws-sns/data/sns-topic-details.png "SNS - Topic details")

3. Click on the dropdown **Other topic actions** and select **Edit topic policy**.

4. From the **Edit topic policy** screen make sure you are in the **Basic View** and modify the options as per the picture below and click **Update**. Basically we allow everyone to become publisher of the topic but only we (the owner) can subscribe to it.

 ![SNS - Edit topic policy](/articles/create-contact-form-using-aws-sns/data/sns-topic-policy-everyone.png "SNS - Edit topic policy")

5. Now you successfully have a topic that anyone can publish messages to.

6. Time to add our email as a subscriber to the topic. From the **Topic Details** page, click on the **Create Subscription** button.

7. Make sure to change the **Protocol** to **Email** and fill out your email on the corresponding box and click **Create Subscription**. An email will be sent to your email in order for you to confirm that you would like to subscribe to this topic.

 ![SNS - Create Topic Subscription](/articles/create-contact-form-using-aws-sns/data/sns-subscription.png "SNS - Create Topic Subscription")

Perfect, we are almost there!

### Code

Here I will provide the very minimal code required to publish a message from your website. I will not include any styling or validation or anything since the scope of the article is just to connect the different AWS services together.

Below you can find the whole HTML and Javascript code used to send a message to your email from the website.


```Javascript
<html>
<head>
 <script src="aws-sdk/dist/aws-sdk.min.js"></script>
 <script type="text/javascript">
 var LPAWS = {};

 // Initialize the Amazon Cognito credentials provider
 AWS.config.region = 'eu-west-1'; // Region
 AWS.config.credentials = new AWS.CognitoIdentityCredentials({
 IdentityPoolId: 'eu-west-1:e4c24108-5050-42f8-ac0b-761c46aa947f',
 });

 LPAWS.sendToTopic = function() {
 var sns = new AWS.SNS();
 var params = {
 //Message: 'Hello topic', /* required */
 Message: document.querySelector('#input-msg').value,
 Subject: 'Browser SNS publish - contact form',
 TopicArn: 'arn:aws:sns:eu-west-1:717437904155:com-website-contact-form'
 };
 sns.publish(params, function(err, data) {
 if (err) console.log(err, err.stack); // an error occurred
 else console.log(data); // successful response
 });
 };
 </script>
</head>
<body>
 <form>
 <input type="text" name="msg" id="input-msg"/>
 <button onclick="LPAWS.sendToTopic(); return false;">Send to SNS topic</button>
 </form>
</body>
</html>

```

I deployed locally the above code and as you can see from the screenshot below it works perfectly! Good for us!

![End-to-End test working](/articles/create-contact-form-using-aws-sns/data/website-test-everyone.png "End-to-End test working")


### Secure our SNS topic

The only issue we have right now is that anyone can publish messages to the SNS topic. Therefore we could receive a lot of spamming if someone inspected our Javascript code and extracted the endpoints for the SNS topic and Cognito identity pool.

The solution to this problem is to add a **policy rule** in our SNS topic to only allow our website to act as a publisher, which is trivial.

1. Login to the SNS console ([direct link](https://console.aws.amazon.com/sns/))

2. From the left side menu select **Topics**

3. From the list of topics select the one we created before, then click at **Actions** and from the dropdown select **Edit topic policy**.

 ![SNS - Secure policy](/articles/create-contact-form-using-aws-sns/data/sns-secure-edit-policy.png "SNS - Secure policy")

4. Now instead of the Basic View you should go to the **Advanced View**

5. In **Advanced View** you can directly edit the topic policy rules in detail. We want to restrict access for publishing only to our domain. Make sure that the **second** statement of the rules is as the following excerpt.

 ```JSON
 {
 "Sid": "__console_pub_0",
 "Effect": "Allow",
 "Principal": {
 "AWS": "*"
 },
 "Action": "SNS:Publish",
 "Resource": "arn:aws:sns:eu-west-1:717437904155:com-website-contact-form",
 "Condition": {
 "StringLike": {
 "aws:Referer": [
 "https://www.lambrospetrou.com/*",
 "https://lambrospetrou.com/*"
 ]
 }
 }
 }
 ```

 The above configuration adds the **Condition** to the existing rule which is what is restricting access to the **Publish** permissions of the topic only to our domain. Make sure that you use your own **Resource arn url** which can be found in the topics details page and also your own domains.

6. Done. Your website is the only one with rights to publish messages into your SNS topic (except of course yourself through the console). Please be advised though that it is not difficult for programmers to create TCP packets with the proper headers to fool this policy. At least we make it a bit harder for the usual user :)


### Conclusion

In conclusion, we have seen how easy it is with **Simple Notification Service (SNS)** and **Amazon Cognito** services to create our own mailing system. You can extend this configuration to send SMS, Push notifications to your devices or anything else you can imagine just by playing with the different SNS subscription types.

There are many features built into SNS that you can use in order to make your messages more advanced and more powerful. You can read all about them in the [AWS SNS documentation](http://aws.amazon.com/documentation/sns/).

If you spot any errors in the tutorial or your experience is different from what is described here please don't hesitate to contact me.

**Disclaimer**

The details used above for the SNS topic and Cognito identity pool were deleted after completing this tutorial.

A common feature among most websites is a way to contact the owner for any comments or feedback (or complaints). In previous versions of my website and blog I used several techniques to provide a contact me form. In almost all the cases I had to implement the actual email submission on the website’s hosting server or in some cases used a 3rd party service that provided me this functionality by querying an HTTP endpoint they gave me with the actual message and they sent the email for me, to me.
Today I decided to roll out my own contact-form service using AWS. As it turns out, it is amazingly easy to setup and it only takes you 15 minutes from the moment you sign up for the free-tier account to the point where you have an active form sending you emails. Awesome time we live in!
<h2 id="assumptions">Assumptions</h2>
<ul>
<li>You have an active AWS account (if not do yourself a favor and register for the 1-year free tier at <a href="https://aws.amazon.com/">AWS</a>).</li>
<li>You have a website somewhere or you know how to use Simple Storage Service (S3) from AWS to host your website files.</li>
</ul>
<h2 id="services">Services</h2>
In order to implement the contact form we are gong to use two services from AWS.
<ol>
<li>Simple Notification Service (SNS) (<a href="https://aws.amazon.com/sns/">SNS service</a>)
 Amazon SNS is a fast, flexible, fully managed pub-sub messaging service. Use it as a cloud-based mobile app notification service to send push notifications, email, and SMS messages; or as an enterprise-messaging infrastructure. (description as per AWS documentation)
</li>
<li>Amazon Cognito (<a href="https://aws.amazon.com/cognito/">Cognito service</a>)
 Amazon Cognito gives you unique identifiers for your end users and then lets you securely store and sync user app data in the AWS Cloud across multiple devices and OS platforms. You can do this with just a few lines of code, and your app can work the same, regardless of whether a user’s devices are online or offline. When new data is available in the sync store, a user’s devices can be alerted by a silent push notification so that your app can sync the new data automatically. (description as per AWS documentation)
</li>
</ol>
<h2 id="procedure">Procedure</h2>
The whole procedure is very simple. We will create an SNS topic (imagine it as an endpoint) where the website will send the message of the contact form. As soon as SNS receives the message it will automatically send it over to our email, which we will have configured through the AWS console to be a subscriber of that specific topic. Amazon Cognito is used just as a way to get temporary credentials for the website in order to be able to publish the message to the SNS topic.
I will explain the implementation in three steps, a) Amazon Cognito, b) SNS and c) Securing our SNS topic to allow only publishings from our website domain.
<h3 id="amazon-cognito">Amazon Cognito</h3>
We use Cognito just for the sake of getting temporary credentials for our website in order to be able to use any AWS service through the Javascript SDK.
<ol>
<li>Login to the AWS console and open the Amazon Cognito service (<a href="https://console.aws.amazon.com/cognito/">direct link</a>)
</li>
<li>Select Create new identity pool
 <img src="/articles/create-contact-form-using-aws-sns/data/cognito-create-pool.png" alt="Amazon Cognito - Create new identity pool" title="Amazon Cognito - Create new identity pool"/>
</li>
<li>Type in a name for your pool (i.e. Website_Contact_Form) and enable Unauthenticated identities.
</li>
<li>You should have something along the lines of the following picture.
 <img src="/articles/create-contact-form-using-aws-sns/data/cognito-info-pool.png" alt="Amazon Cognito - Identity pool information" title="Amazon Cognito - Identity pool information"/>
</li>
<li>Click Create Pool and then on the next page you get the message “Your Cognito identities require access to your resources”. Just click Allow (bottom right) to proceed.
</li>
<li>You should be in the Edit identity pool page now where you see the information you provided above. From the left side menu choose Sample code and in the Platform dropdown select Javascript.
</li>
<li>Download the AWS SDK since you will use it on your website later.
</li>
<li>You can use the code provided to get temporary credentials for the newly created identity pool. In my case the code is similar to what is shown below.
<pre><code class="language-Javascript"> // Initialize the Amazon Cognito credentials provider
 AWS.config.region = &#39;eu-west-1&#39;; // Region
 AWS.config.credentials = new AWS.CognitoIdentityCredentials({
 IdentityPoolId: &#39;eu-west-1:e4c24108-5050-42f8-ac0b-761c46aa947f&#39;,
 });</code></pre>
</li>
</ol>
That’s it for the Cognito service!
<h3 id="simple-notification-service-sns">Simple Notification Service (SNS)</h3>
SNS is a bit longer procedure but nothing complex. The philosophy behind SNS is that you have a topic (endpoint) with some publishers and some subscribers. When a publisher posts something to that topic then all the subscribers are notified automatically by the service with the newly published message. I guess by now you should have figured out that the subscriber to our SNS topic will be our email and the publisher of the topic will be our website.
Let’s dive in!
<ol>
<li>Login to the AWS console and open the SNS service (<a href="https://console.aws.amazon.com/sns/">direct link</a>)
 <img src="/articles/create-contact-form-using-aws-sns/data/sns-create-topic.png" alt="SNS - Create Topic" title="SNS - Create Topic"/>
</li>
<li>Select Create Topic and then just fill out the Topic name with whatever you want (i.e. com-website-contact-form). You should now see the Topic Details page for your new topic (similar to the picture below).
 <img src="/articles/create-contact-form-using-aws-sns/data/sns-topic-details.png" alt="SNS - Topic details" title="SNS - Topic details"/>
</li>
<li>Click on the dropdown Other topic actions and select Edit topic policy.
</li>
<li>From the Edit topic policy screen make sure you are in the Basic View and modify the options as per the picture below and click Update. Basically we allow everyone to become publisher of the topic but only we (the owner) can subscribe to it.
 <img src="/articles/create-contact-form-using-aws-sns/data/sns-topic-policy-everyone.png" alt="SNS - Edit topic policy" title="SNS - Edit topic policy"/>
</li>
<li>Now you successfully have a topic that anyone can publish messages to.
</li>
<li>Time to add our email as a subscriber to the topic. From the Topic Details page, click on the Create Subscription button.
</li>
<li>Make sure to change the Protocol to Email and fill out your email on the corresponding box and click Create Subscription. An email will be sent to your email in order for you to confirm that you would like to subscribe to this topic.
 <img src="/articles/create-contact-form-using-aws-sns/data/sns-subscription.png" alt="SNS - Create Topic Subscription" title="SNS - Create Topic Subscription"/>
</li>
</ol>
Perfect, we are almost there!
<h3 id="code">Code</h3>
Here I will provide the very minimal code required to publish a message from your website. I will not include any styling or validation or anything since the scope of the article is just to connect the different AWS services together.
Below you can find the whole HTML and Javascript code used to send a message to your email from the website.
<pre><code class="language-Javascript">&lt;html&gt;
&lt;head&gt;
 &lt;script src=&quot;aws-sdk/dist/aws-sdk.min.js&quot;&gt;&lt;/script&gt;
 &lt;script type=&quot;text/javascript&quot;&gt;
 var LPAWS = {};

 // Initialize the Amazon Cognito credentials provider
 AWS.config.region = &#39;eu-west-1&#39;; // Region
 AWS.config.credentials = new AWS.CognitoIdentityCredentials({
 IdentityPoolId: &#39;eu-west-1:e4c24108-5050-42f8-ac0b-761c46aa947f&#39;,
 });

 LPAWS.sendToTopic = function() {
 var sns = new AWS.SNS();
 var params = {
 //Message: &#39;Hello topic&#39;, /* required */
 Message: document.querySelector(&#39;#input-msg&#39;).value,
 Subject: &#39;Browser SNS publish - contact form&#39;,
 TopicArn: &#39;arn:aws:sns:eu-west-1:717437904155:com-website-contact-form&#39;
 };
 sns.publish(params, function(err, data) {
 if (err) console.log(err, err.stack); // an error occurred
 else console.log(data); // successful response
 });
 };
 &lt;/script&gt;
&lt;/head&gt;
&lt;body&gt;
 &lt;form&gt;
 &lt;input type=&quot;text&quot; name=&quot;msg&quot; id=&quot;input-msg&quot;/&gt;
 &lt;button onclick=&quot;LPAWS.sendToTopic(); return false;&quot;&gt;Send to SNS topic&lt;/button&gt;
 &lt;/form&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre>
I deployed locally the above code and as you can see from the screenshot below it works perfectly! Good for us!
<img src="/articles/create-contact-form-using-aws-sns/data/website-test-everyone.png" alt="End-to-End test working" title="End-to-End test working"/>
<h3 id="secure-our-sns-topic">Secure our SNS topic</h3>
The only issue we have right now is that anyone can publish messages to the SNS topic. Therefore we could receive a lot of spamming if someone inspected our Javascript code and extracted the endpoints for the SNS topic and Cognito identity pool.
The solution to this problem is to add a policy rule in our SNS topic to only allow our website to act as a publisher, which is trivial.
<ol>
<li>Login to the SNS console (<a href="https://console.aws.amazon.com/sns/">direct link</a>)
</li>
<li>From the left side menu select Topics
</li>
<li>From the list of topics select the one we created before, then click at Actions and from the dropdown select Edit topic policy.
 <img src="/articles/create-contact-form-using-aws-sns/data/sns-secure-edit-policy.png" alt="SNS - Secure policy" title="SNS - Secure policy"/>
</li>
<li>Now instead of the Basic View you should go to the Advanced View
</li>
<li>In Advanced View you can directly edit the topic policy rules in detail. We want to restrict access for publishing only to our domain. Make sure that the second statement of the rules is as the following excerpt.
<pre><code class="language-JSON"> {
 &quot;Sid&quot;: &quot;__console_pub_0&quot;,
 &quot;Effect&quot;: &quot;Allow&quot;,
 &quot;Principal&quot;: {
 &quot;AWS&quot;: &quot;*&quot;
 },
 &quot;Action&quot;: &quot;SNS:Publish&quot;,
 &quot;Resource&quot;: &quot;arn:aws:sns:eu-west-1:717437904155:com-website-contact-form&quot;,
 &quot;Condition&quot;: {
 &quot;StringLike&quot;: {
 &quot;aws:Referer&quot;: [
 &quot;https://www.lambrospetrou.com/*&quot;,
 &quot;https://lambrospetrou.com/*&quot;
 ]
 }
 }
 }</code></pre>
 The above configuration adds the Condition to the existing rule which is what is restricting access to the Publish permissions of the topic only to our domain. Make sure that you use your own Resource arn url which can be found in the topics details page and also your own domains.
</li>
<li>Done. Your website is the only one with rights to publish messages into your SNS topic (except of course yourself through the console). Please be advised though that it is not difficult for programmers to create TCP packets with the proper headers to fool this policy. At least we make it a bit harder for the usual user :)
</li>
</ol>
<h3 id="conclusion">Conclusion</h3>
In conclusion, we have seen how easy it is with Simple Notification Service (SNS) and Amazon Cognito services to create our own mailing system. You can extend this configuration to send SMS, Push notifications to your devices or anything else you can imagine just by playing with the different SNS subscription types.
There are many features built into SNS that you can use in order to make your messages more advanced and more powerful. You can read all about them in the <a href="http://aws.amazon.com/documentation/sns/">AWS SNS documentation</a>.
If you spot any errors in the tutorial or your experience is different from what is described here please don’t hesitate to contact me.
Disclaimer
The details used above for the SNS topic and Cognito identity pool were deleted after completing this tutorial.

Guide that describes how you create a contact form for your website using Amazon Web Services (AWS) Simple Notification Service (SNS) without any server.

Create a contact form using Amazon Web Services (AWS) Simple Notification Service (SNS)


Time passes and goes at amazing speed. Finally, after twenty-five years, school has reached its end. I still don't know if this is bad or good, since I am now thrown into the wild jungle known as life.

Anyway, enough with the philosophical talk. Now that I graduated from university I have some time to revisit some of my old projects and refactor them. As always, when I want to try something new I re-write my website. During last week I completely re-wrote my website and blog from scratch, ```lambrospetrou.com```.

As I described in my last article, dated in 2014-July-20, I am in love with Go aka [#golang](http://golang.org) so it was a natural choice to use it again. I decided to go away from the dynamic nature of a blog and revert to a static blog with dynamic content ;) Well, let's just say that whenever I create write an article I regenerate the website. Static Site Generators have seen a lot of popularity increase in the last couple of months, maybe because they are easy to use or because the site is easier to deploy than messing around with Wordpress, Joomla, plugins and all the sh\*\*t brought with them. Additionally, as a Software Engineer I am always fond of my own creations. You can find the generator for this blog at Github: [Micro-blog generator repository](https://github.com/lambrospetrou/gomicroblog). I have to warn you that it is not exactly designed for generic usage. However, if you like its simplicity and want to try it just check out the source code of this blog at [Lambros Petrou blog source code](https://github.com/lambrospetrou/lambrospetrou.github.io) and replace it with your own content.

Apart from my website and blog, another important project I had in pending state was [Spi.To](http://spi.to), my own custom URL shortener and text share tool. During last year, I was experimenting with Polymer and Paper elements to give it a material design makeover. However, I found it to be pretty unstable (it was still in development stages so it might have fixed its issues with its current stable version) and very heavy for the simple and minimalistic product I wanted. I ported most of the stuff I needed into native [Dart](https://www.dartlang.org/) and now I find it to be much more responsive.

I plan to add more features to Spi.to during the coming months and I really want to replace the back-end with a server-less infrastructure at Amazon Web Services (AWS). **AWS** is among my top priorities in the **To-Learn** list for this year and I would love to create more projects utilizing the myriad services available by Amazon, especially the new **API Gateway**, **Lambda functions** and of course the awesome **Simple Storage Service (S3)**.

One thing I wanted to do for a long time was to find a way to edit my CV (resume) easily without booting into Windows just to use Microsoft Word and without searching for a Latex template that would most probably not match the design I wanted. I decided to turn to the easiest platform for customizations and easy updates, the web. I replicated the design of my old CV using only HTML and CSS, leading to a pixel-perfect document which is amazingly easy to update and to customize in-detail. You can find my new CV version at [lambrospetrou.com/cv/](https://lambrospetrou.com/cv/).

In conclusion, I would like to take advantage of my time working at Amazon and be certified as an [AWS Certified Professional](https://aws.amazon.com/certification/), which will boost my knowledge and skills around cloud computing and at the same time open many more doors career-wise. As it seems **AWS** is the dominant cloud computing provider and it has no plans to cease existing for the foreseeable future.

It is nice to be in production mode again :)


Time passes and goes at amazing speed. Finally, after twenty-five years, school has reached its end. I still don’t know if this is bad or good, since I am now thrown into the wild jungle known as life.
Anyway, enough with the philosophical talk. Now that I graduated from university I have some time to revisit some of my old projects and refactor them. As always, when I want to try something new I re-write my website. During last week I completely re-wrote my website and blog from scratch, <code>lambrospetrou.com</code>.
As I described in my last article, dated in 2014-July-20, I am in love with Go aka <a href="http://golang.org">#golang</a> so it was a natural choice to use it again. I decided to go away from the dynamic nature of a blog and revert to a static blog with dynamic content ;) Well, let’s just say that whenever I create write an article I regenerate the website. Static Site Generators have seen a lot of popularity increase in the last couple of months, maybe because they are easy to use or because the site is easier to deploy than messing around with Wordpress, Joomla, plugins and all the sh**t brought with them. Additionally, as a Software Engineer I am always fond of my own creations. You can find the generator for this blog at Github: <a href="https://github.com/lambrospetrou/gomicroblog">Micro-blog generator repository</a>. I have to warn you that it is not exactly designed for generic usage. However, if you like its simplicity and want to try it just check out the source code of this blog at <a href="https://github.com/lambrospetrou/lambrospetrou.github.io">Lambros Petrou blog source code</a> and replace it with your own content.
Apart from my website and blog, another important project I had in pending state was <a href="http://spi.to">Spi.To</a>, my own custom URL shortener and text share tool. During last year, I was experimenting with Polymer and Paper elements to give it a material design makeover. However, I found it to be pretty unstable (it was still in development stages so it might have fixed its issues with its current stable version) and very heavy for the simple and minimalistic product I wanted. I ported most of the stuff I needed into native <a href="https://www.dartlang.org/">Dart</a> and now I find it to be much more responsive.
I plan to add more features to Spi.to during the coming months and I really want to replace the back-end with a server-less infrastructure at Amazon Web Services (AWS). AWS is among my top priorities in the To-Learn list for this year and I would love to create more projects utilizing the myriad services available by Amazon, especially the new API Gateway, Lambda functions and of course the awesome Simple Storage Service (S3).
One thing I wanted to do for a long time was to find a way to edit my CV (resume) easily without booting into Windows just to use Microsoft Word and without searching for a Latex template that would most probably not match the design I wanted. I decided to turn to the easiest platform for customizations and easy updates, the web. I replicated the design of my old CV using only HTML and CSS, leading to a pixel-perfect document which is amazingly easy to update and to customize in-detail. You can find my new CV version at <a href="https://lambrospetrou.com/cv/">lambrospetrou.com/cv/</a>.
In conclusion, I would like to take advantage of my time working at Amazon and be certified as an <a href="https://aws.amazon.com/certification/">AWS Certified Professional</a>, which will boost my knowledge and skills around cloud computing and at the same time open many more doors career-wise. As it seems AWS is the dominant cloud computing provider and it has no plans to cease existing for the foreseeable future.
It is nice to be in production mode again :)

School is finally history freeing up more time to do exciting and interesting things.

After-school effect


I have been a user of Wordpress since 2006 in order to provide my own blog, latest version found at [MasterGenius.NET](http://mastergenius.net). I invested a lot of time in it learning PHP, web development (HTML, CSS, Javascript and the likes) and I thank it since it made my way into the web as a developer. There have been many times I wanted to switch out from it and make my own micro-blog service but due to time constraints and other things popping up I always halted the process.

Recently, I started using Go language as my primary tool for my latest projects and I am really amazed by how much you can create in a small amount of time and at the same time having a substantial performance. All other languages that tried to do something similar failed either in performance or in coding-style, at least for my taste. **Go is the sweet spot** ;)

I wanted a project in order to learn Go further than reading and doing some tutorials so the micro-blog service project kicked-off. This blog is the result of those 3-4 days and I am going to use it as a more personal blog with thoughts and opinions on variety of topics and also post some useful tutorials (without going too much out-of-topic though like my previous blog).

This blog is written entirely in pure **Go** [#golang](http://golang.org) language without any 3rd party library (i.e gorilla/mux or martini) that many Go articles suggested and I have to admit that I am very impressed. The core standard library is very sufficient and provides you all the tools to create anything you want easily. As a back-end I am using **Couchbase server 2.x** which is one of the best NoSQL databases I have tried so far, and speedy-demon too. As for the content of the articles I made a very minimalistic and simple panel where I can write them in **Markdown** format. I have chosen Markdown for the articles because it lets me write in plain text without any HTML tags or CSS classes getting involved, thus I can concentrate on the content. And it is good to know since it is being used in many other services too, like *Github*. The Markdown content is compiled in HTML when posted by the Go back-end.

The old blog is still live, at least for now, and I will try to find some noteworthy tutorials I wrote back in the days and post them here too, like I already did with some of them but I will keep this blog cleaner and more personal.

I hope you like this new blog and if you have any suggestions don't hesitate to contact me ;)


I have been a user of Wordpress since 2006 in order to provide my own blog, latest version found at <a href="http://mastergenius.net">MasterGenius.NET</a>. I invested a lot of time in it learning PHP, web development (HTML, CSS, Javascript and the likes) and I thank it since it made my way into the web as a developer. There have been many times I wanted to switch out from it and make my own micro-blog service but due to time constraints and other things popping up I always halted the process.
Recently, I started using Go language as my primary tool for my latest projects and I am really amazed by how much you can create in a small amount of time and at the same time having a substantial performance. All other languages that tried to do something similar failed either in performance or in coding-style, at least for my taste. Go is the sweet spot ;)
I wanted a project in order to learn Go further than reading and doing some tutorials so the micro-blog service project kicked-off. This blog is the result of those 3-4 days and I am going to use it as a more personal blog with thoughts and opinions on variety of topics and also post some useful tutorials (without going too much out-of-topic though like my previous blog).
This blog is written entirely in pure Go <a href="http://golang.org">#golang</a> language without any 3rd party library (i.e gorilla/mux or martini) that many Go articles suggested and I have to admit that I am very impressed. The core standard library is very sufficient and provides you all the tools to create anything you want easily. As a back-end I am using Couchbase server 2.x which is one of the best NoSQL databases I have tried so far, and speedy-demon too. As for the content of the articles I made a very minimalistic and simple panel where I can write them in Markdown format. I have chosen Markdown for the articles because it lets me write in plain text without any HTML tags or CSS classes getting involved, thus I can concentrate on the content. And it is good to know since it is being used in many other services too, like Github. The Markdown content is compiled in HTML when posted by the Go back-end.
The old blog is still live, at least for now, and I will try to find some noteworthy tutorials I wrote back in the days and post them here too, like I already did with some of them but I will keep this blog cleaner and more personal.
I hope you like this new blog and if you have any suggestions don’t hesitate to contact me ;)

I switched from Wordpress to my custom own-made blog coded in Go-lang.

Micro-blog service written in Go as a personal blog



If you have ever installed wordpress on your own server, at some time you faced permissions problems. Especially when you are migrating a wordpress blog from one host to another and there is file copying involvement through ftp, scp etc.

If not, lucky you! :)

## Scenario

| Description | Value |
|-------------|------|
| username | lpuser |
| apache2 groupname | www-data |
| wordpress dir | /home/lpuser/wordpress/ |

Let's say you are ```lpuser``` and your server that runs the wordpress blog belongs to the ```www-data``` permissions group.

## Wordpress Permissions

Everything should have the ```0755``` permission for the user, apart from the *.htaccess* files and the *wp-admin/index.php* file that need to have ```0644```.

Let's do this:

```bash
sudo chmod -R 0755 /home/lpuser/wordpress/
sudo chmod 0644 /home/lpuser/wordpress/.htaccess
sudo chmod 0644 /home/lpuser/wordpress/wp-admin/index.php
```

Now if you execute the following command you should verify that everything has ```rwxr-xr-x```, and the two other files ```rw-r--r--```.

If you try now to make changes to your theme through the wordpress editor then you won't be able to save since the server does not have write permissions.
We can find that apache runs under the user ```www-data``` and the same group, as seen by the following commands:

```bash
ps aux | grep apache
groups www-data
```

So an easy solution to fix the problems is to add ourselves to the group ```www-data``` and then make the files to be owned by us, and the new group we belong.

This can be achieved with the following commands:

```bash
sudo usermod -a -G www-data lpuser
sudo chown -R lpuser:www-data /home/lpuser/wordpress/wp-content/themes
chmod -R 0775 /home/lpuser/wordpress/wp-content/themes
```

The last command allows the group to read/write/execute on the **themes** folder only.

If you have another way don't hesitate to share it. The scenario is that we have a username different than the server's that runs the wordpress blog.


If you have ever installed wordpress on your own server, at some time you faced permissions problems. Especially when you are migrating a wordpress blog from one host to another and there is file copying involvement through ftp, scp etc.
If not, lucky you! :)
<h2 id="scenario">Scenario</h2>
<table>
<thead>
<tr>
<th>Description</th>
<th>Value</th>
</tr>
</thead>
<tbody><tr>
<td>username</td>
<td>lpuser</td>
</tr>
<tr>
<td>apache2 groupname</td>
<td>www-data</td>
</tr>
<tr>
<td>wordpress dir</td>
<td>/home/lpuser/wordpress/</td>
</tr>
</tbody></table>
Let’s say you are <code>lpuser</code> and your server that runs the wordpress blog belongs to the <code>www-data</code> permissions group.
<h2 id="wordpress-permissions">Wordpress Permissions</h2>
Everything should have the <code>0755</code> permission for the user, apart from the .htaccess files and the wp-admin/index.php file that need to have <code>0644</code>.
Let’s do this:
<pre><code class="language-bash">sudo chmod -R 0755 /home/lpuser/wordpress/
sudo chmod 0644 /home/lpuser/wordpress/.htaccess
sudo chmod 0644 /home/lpuser/wordpress/wp-admin/index.php</code></pre>
Now if you execute the following command you should verify that everything has <code>rwxr-xr-x</code>, and the two other files <code>rw-r--r--</code>.
If you try now to make changes to your theme through the wordpress editor then you won’t be able to save since the server does not have write permissions.
We can find that apache runs under the user <code>www-data</code> and the same group, as seen by the following commands:
<pre><code class="language-bash">ps aux | grep apache
groups www-data</code></pre>
So an easy solution to fix the problems is to add ourselves to the group <code>www-data</code> and then make the files to be owned by us, and the new group we belong.
This can be achieved with the following commands:
<pre><code class="language-bash">sudo usermod -a -G www-data lpuser
sudo chown -R lpuser:www-data /home/lpuser/wordpress/wp-content/themes
chmod -R 0775 /home/lpuser/wordpress/wp-content/themes</code></pre>
The last command allows the group to read/write/execute on the themes folder only.
If you have another way don’t hesitate to share it. The scenario is that we have a username different than the server’s that runs the wordpress blog.

Fix the permissions of your own wordpress installation.

Wordpress permissions guide


Many users are dual booting Windows and Ubuntu for numerous reasons today. Well a problem that may arise is that if you have Dropbox, you should have 2 folders of your Dropbox files, 1 for Windows and 1 for Ubuntu.

Following the guide below you can consolidate them into one and reclaim your wasted space back.

**NOTE**: I am assuming that the Windows partition is mounted at startup when you boot into Ubuntu.

## Guidelines

1. Set up your Dropbox folder inside **Windows** first and verify that it is syncing your files without problems.

2. Boot into Ubuntu and stop Dropbox from syncing.

    ```bash
    pkill dropbox
    ```

3. Delete your Dropbox folder from Ubuntu (the command below assumes it's located in your home directory).

    ```bash
    rm -rf /home/_username_/Dropbox/
    ```

4. Create a symlink to your Windows Dropbox folder (this will create a symbolic link  in your home directory that links to your Dropbox folder in Windows).

    ```bash
    ln -s /media/WindowsOS/User/_username_/Dropbox /home/_username_/Dropbox
    ```

    ```/media/WindowsOS/User/_username_/Dropbox```
    the path to the Dropbox folder in your Windows partition mounted in Ubuntu  

    ```/home/_username_/Dropbox```
    the default folder for Dropbox to synchronize

5. Reboot and you’ re finished. Now you maintain only one copy of your Dropbox folder which needs to be synced only once, no matter which OS is booted at any time.

That's it, I hope you won't find any problems. If you do though, contact me :)


Many users are dual booting Windows and Ubuntu for numerous reasons today. Well a problem that may arise is that if you have Dropbox, you should have 2 folders of your Dropbox files, 1 for Windows and 1 for Ubuntu.
Following the guide below you can consolidate them into one and reclaim your wasted space back.
NOTE: I am assuming that the Windows partition is mounted at startup when you boot into Ubuntu.
<h2 id="guidelines">Guidelines</h2>
<ol>
<li>Set up your Dropbox folder inside Windows first and verify that it is syncing your files without problems.
</li>
<li>Boot into Ubuntu and stop Dropbox from syncing.
<pre><code class="language-bash"> pkill dropbox</code></pre>
</li>
<li>Delete your Dropbox folder from Ubuntu (the command below assumes it’s located in your home directory).
<pre><code class="language-bash"> rm -rf /home/_username_/Dropbox/</code></pre>
</li>
<li>Create a symlink to your Windows Dropbox folder (this will create a symbolic link in your home directory that links to your Dropbox folder in Windows).
<pre><code class="language-bash"> ln -s /media/WindowsOS/User/_username_/Dropbox /home/_username_/Dropbox</code></pre>
 <code>/media/WindowsOS/User/_username_/Dropbox</code>
 the path to the Dropbox folder in your Windows partition mounted in Ubuntu 
 <code>/home/_username_/Dropbox</code>
 the default folder for Dropbox to synchronize
</li>
<li>Reboot and you’ re finished. Now you maintain only one copy of your Dropbox folder which needs to be synced only once, no matter which OS is booted at any time.
</li>
</ol>
That’s it, I hope you won’t find any problems. If you do though, contact me :)

Tutorial on how to share your Dropbox sync folder between  Windows and Linux.

Share your Dropbox folder between Windows and Ubuntu on dual-boot system


Since version 7 of Java Ubuntu no longer has in its packages the required files for the Oracle JDK 7 due to new license terms by Oracle. The default option for Ubuntu is OpenJDK but if you really want the standard Java from Oracle (former Sun) then you have no easy way to do it on your own.

Therefore I prepared a simple script that will install the Java 7 development toolkit on your system with no effort at all.

1. Download the correct package from Oracle website: [JavaSE Downloads](http://www.oracle.com/technetwork/java/javase/downloads/index.html "JavaSE Downloads").  
Choose the JDK version and afterwards **Linux x86** or **Linux x64** depending on your system. It must be the file with the **.tar.gz** extension.

2. Download my script from here: [Oracle-JDK-Installer Repository](https://github.com/lambrospetrou/oracle-jdk-installer)  
Click the button **ZIP** and after downloading it extract it anywhere, and remember the path.

3. Put the package you downloaded at step 1 in the same directory as the **jdk-installer.sh** you downloaded (step 2).

4. Type

    ```bash
    sudo ./jdk-installer.sh PATH_TO_PACKAGE SYSTEM_ARCHITECTURE
    ```
You can read the instructions at my github repository (step 2) for more information.

5. Verify your installation:

    ```bash
    java -version
    ```
    You should get a message mentioning the version you downloaded.

6. Visit [Verify Java Version](http://www.java.com/en/download/installed.jsp) and click **Verify Java version** in order to check that the browser plugin works.

That's it folks!


Since version 7 of Java Ubuntu no longer has in its packages the required files for the Oracle JDK 7 due to new license terms by Oracle. The default option for Ubuntu is OpenJDK but if you really want the standard Java from Oracle (former Sun) then you have no easy way to do it on your own.
Therefore I prepared a simple script that will install the Java 7 development toolkit on your system with no effort at all.
<ol>
<li>Download the correct package from Oracle website: <a href="http://www.oracle.com/technetwork/java/javase/downloads/index.html" title="JavaSE Downloads">JavaSE Downloads</a>. 
Choose the JDK version and afterwards Linux x86 or Linux x64 depending on your system. It must be the file with the .tar.gz extension.
</li>
<li>Download my script from here: <a href="https://github.com/lambrospetrou/oracle-jdk-installer">Oracle-JDK-Installer Repository</a> 
Click the button ZIP and after downloading it extract it anywhere, and remember the path.
</li>
<li>Put the package you downloaded at step 1 in the same directory as the jdk-installer.sh you downloaded (step 2).
</li>
<li>Type
<pre><code class="language-bash"> sudo ./jdk-installer.sh PATH_TO_PACKAGE SYSTEM_ARCHITECTURE</code></pre>
You can read the instructions at my github repository (step 2) for more information.
</li>
<li>Verify your installation:
<pre><code class="language-bash"> java -version</code></pre>
 You should get a message mentioning the version you downloaded.
</li>
<li>Visit <a href="http://www.java.com/en/download/installed.jsp">Verify Java Version</a> and click Verify Java version in order to check that the browser plugin works.
</li>
</ol>
That’s it folks!

Guide explaining how to easily install Java JDK on your Ubuntu 12.04 LTS.

Install JDK 7 (Java Development Toolkit) on Ubuntu 12.04 LTS


After researching for a few hours online to find a guide on how to control my Thinkpad's fan speed I realized that the new models have some differences from previous models and the guides available are not complete if not wrong. So, I am making this tutorial for anyone that has a new Thinkpad (x30/x20 models) and needs to control his fan in order to keep the noise down and get more battery life.

Every step below uses the terminal so open an instance with the combination ```CRTL + ALT + T```

1. The first thing we will do is to install a program that will provide us information about the sensors of the laptop and their temperatures

    ```bash
    sudo apt-get install lm-sensors
    ```
Configure the application in order to find every available sensor

    ```bash
    sudo sensors-detect
    ```
Answer **Yes** to every question and the last confirmation for saving the changes made.

2. Install **thinkfan** which is our main program:

    ```bash
    sudo apt-get install thinkfan
    ```

3. Add the **coretemp** module to the startup list. It will provide us the temperature inputs.

    ```bash
    echo coretemp >> /etc/modules
    ```

4. Load the **coretemp** module.

    ```bash
    sudo modprobe coretemp
    ```

5. The next step is to find your temperature inputs so take note the results of the following command.

    ```bash
    sudo find /sys/devices -type f -name "temp*_input"
    ```

    If you don't get any outputs ( similar to the next step ) please Reboot and continue from this step.

6. It's time to edit our **thinkfan** configuration

    ```bash
    sudo vim /etc/thinkfan.conf
    ```

    Go to the line where it says **#sensor /proc/acpi/ibm/thermal ...** and below that line (which should be commented since thermal is not supported in the new thinkpads) insert something like the following:

    ```bash
    sensor /sys/devices/platform/coretemp.0/temp1_input
    sensor /sys/devices/platform/coretemp.0/temp2_input
    sensor /sys/devices/platform/coretemp.0/temp3_input
    sensor /sys/devices/virtual/hwmon/hwmon0/temp1_input
    ```
    The above lines are the results from Step 5 prefixed with 'sensor '.

7. Time to set the temperature rules. The format is: ```FAN_LEVEL, LOW_TEMP, HIGH_TEMP``` meaning that each *FAN_LEVEL* will start when the highest temperature reported by all the sensors meets its *LOW_TEMP* and if it surpasses its *HIGH_TEMP* it will go to the next *FAN_LEVEL* rule. If it goes below the *LOW_TEMP* it will fallback to the previous *FAN_LEVEL* rule.

    **Please take notice** that the *HIGH_TEMP* of a rule must be between the *LOW_TEMP* & *HIGH_TEMP* of the rule that follows.

    #### My settings are:

    ```
    #(FAN_LEVEL, LOW, HIGH)
    (0,    0,    60)
    (1,    57,    63)
    (2,    60,    66)
    (3,    64,    68)
    (4,    66,    72)
    (5,    70,    74)
    (7,    72,    32767)
    ```
    **NOTE: I am not responsible for any problems you encounter with these rules. They are fine as per my configuration so please test them before using them and if necessary adjust them to your needs.**

8. Now, we must add a configuration file into the modprobe.d

    ```bash
    sudo echo "options thinkpad_acpi fan_control=1" >> /etc/modprobe.d/thinkpad.conf
    ```

9. If you want to start **thinkfan** automatically at boot-time please type the following:

    ```bash
    sudo vim /etc/default/thinkfan
    ```

    Change the line **START=no** to **START=yes**. If the line does not exist, add it yourself.

10. **RESTART** your laptop and everything should work as expected. Test your laptop's temperatures (using **sensors** command) under different workloads and verify that the fan speed is as per the rules you defined.

If you encounter a typing mistake or a step not working for you please contact me.


After researching for a few hours online to find a guide on how to control my Thinkpad’s fan speed I realized that the new models have some differences from previous models and the guides available are not complete if not wrong. So, I am making this tutorial for anyone that has a new Thinkpad (x30/x20 models) and needs to control his fan in order to keep the noise down and get more battery life.
Every step below uses the terminal so open an instance with the combination <code>CRTL + ALT + T</code>
<ol>
<li>The first thing we will do is to install a program that will provide us information about the sensors of the laptop and their temperatures
<pre><code class="language-bash"> sudo apt-get install lm-sensors</code></pre>
Configure the application in order to find every available sensor
<pre><code class="language-bash"> sudo sensors-detect</code></pre>
Answer Yes to every question and the last confirmation for saving the changes made.
</li>
<li>Install thinkfan which is our main program:
<pre><code class="language-bash"> sudo apt-get install thinkfan</code></pre>
</li>
<li>Add the coretemp module to the startup list. It will provide us the temperature inputs.
<pre><code class="language-bash"> echo coretemp &gt;&gt; /etc/modules</code></pre>
</li>
<li>Load the coretemp module.
<pre><code class="language-bash"> sudo modprobe coretemp</code></pre>
</li>
<li>The next step is to find your temperature inputs so take note the results of the following command.
<pre><code class="language-bash"> sudo find /sys/devices -type f -name &quot;temp*_input&quot;</code></pre>
 If you don’t get any outputs ( similar to the next step ) please Reboot and continue from this step.
</li>
<li>It’s time to edit our thinkfan configuration
<pre><code class="language-bash"> sudo vim /etc/thinkfan.conf</code></pre>
 Go to the line where it says #sensor /proc/acpi/ibm/thermal … and below that line (which should be commented since thermal is not supported in the new thinkpads) insert something like the following:
<pre><code class="language-bash"> sensor /sys/devices/platform/coretemp.0/temp1_input
 sensor /sys/devices/platform/coretemp.0/temp2_input
 sensor /sys/devices/platform/coretemp.0/temp3_input
 sensor /sys/devices/virtual/hwmon/hwmon0/temp1_input</code></pre>
 The above lines are the results from Step 5 prefixed with ‘sensor ‘.
</li>
<li>Time to set the temperature rules. The format is: <code>FAN_LEVEL, LOW_TEMP, HIGH_TEMP</code> meaning that each FAN_LEVEL will start when the highest temperature reported by all the sensors meets its LOW_TEMP and if it surpasses its HIGH_TEMP it will go to the next FAN_LEVEL rule. If it goes below the LOW_TEMP it will fallback to the previous FAN_LEVEL rule.
 Please take notice that the HIGH_TEMP of a rule must be between the LOW_TEMP &amp; HIGH_TEMP of the rule that follows.
<h4 id="my-settings-are">My settings are:</h4>
<pre><code> #(FAN_LEVEL, LOW, HIGH)
 (0, 0, 60)
 (1, 57, 63)
 (2, 60, 66)
 (3, 64, 68)
 (4, 66, 72)
 (5, 70, 74)
 (7, 72, 32767)</code></pre> NOTE: I am not responsible for any problems you encounter with these rules. They are fine as per my configuration so please test them before using them and if necessary adjust them to your needs.
</li>
<li>Now, we must add a configuration file into the modprobe.d
<pre><code class="language-bash"> sudo echo &quot;options thinkpad_acpi fan_control=1&quot; &gt;&gt; /etc/modprobe.d/thinkpad.conf</code></pre>
</li>
<li>If you want to start thinkfan automatically at boot-time please type the following:
<pre><code class="language-bash"> sudo vim /etc/default/thinkfan</code></pre>
 Change the line START=no to START=yes. If the line does not exist, add it yourself.
</li>
<li>RESTART your laptop and everything should work as expected. Test your laptop’s temperatures (using sensors command) under different workloads and verify that the fan speed is as per the rules you defined.
</li>
</ol>
If you encounter a typing mistake or a step not working for you please contact me.

Easy way to control the fan speed of your Thinkpad based on the temperatures of its sensors.

Control the fan speed of you Thinkpad T430 running Ubuntu 12.04 LTS


Anyone that purchased a laptop with nVidia Optimus is facing problems with Linux support. In a few words, Optimus is not supported officially ( the best project so far is Bumblebee ). As a result the power draw is more than double compared to integrated card-only systems because both cards are enabled and loaded but only one is in use, the Integrated one.

**Bumblebee** is a project that tries to bring Optimus support in Ubuntu and so far it had good results but the battery impact was inevitable.

The latest update though changed that and now linux fans can have their Optimus configuration without the enormous battery draw. Follow the steps below to install bumblebee but please note that I am assuming you removed any nVidia drivers you may installed on your own.

- We must add the repositories for the bumblebee packages so please type the following in your terminal ```CRTL + ALT + T```

    ```bash
    sudo add-apt-repository ppa:ubuntu-x-swat/x-updates
    sudo add-apt-repository ppa:bumblebee/stable
    sudo apt-get update
    ```

- Now install the bumblebee:

    ```bash
    sudo apt-get install bumblebee bumblebee-nvidia
    ```

    **NOTE:** If you do not want the discrete card to be enabled at all please use the following command instead:

    ```bash
    sudo apt-get install --no-install-recommends bumblebee
    ```

    This will not install the drivers for your discrete card  but if you want in the future to have it just enter the following and get it installed:

    ```bash
    sudo apt-get install bumblebee-nvidia
    ```

- Some people recommend to *reboot* now although I saw the results without doing so.

- If you installed the nVidia driver too, in order to launch a program and use the discrete card you must use the following command:

    ```bash
    optirun _appnamehere_ &
    ```

In order to get support for 32-bit applications with nVidia you should do the following:

```bash
sudo apt-get install virtualgl-libs-ia32
```

I hope that you will find this guide helpful. I got amazing results with it myself:  
Power draw before: 26-28w  
Power draw after: 11-13w

[Bumblebee Project Website](http://bumblebee-project.org/)


Anyone that purchased a laptop with nVidia Optimus is facing problems with Linux support. In a few words, Optimus is not supported officially ( the best project so far is Bumblebee ). As a result the power draw is more than double compared to integrated card-only systems because both cards are enabled and loaded but only one is in use, the Integrated one.
Bumblebee is a project that tries to bring Optimus support in Ubuntu and so far it had good results but the battery impact was inevitable.
The latest update though changed that and now linux fans can have their Optimus configuration without the enormous battery draw. Follow the steps below to install bumblebee but please note that I am assuming you removed any nVidia drivers you may installed on your own.
<ul>
<li>We must add the repositories for the bumblebee packages so please type the following in your terminal <code>CRTL + ALT + T</code>
<pre><code class="language-bash"> sudo add-apt-repository ppa:ubuntu-x-swat/x-updates
 sudo add-apt-repository ppa:bumblebee/stable
 sudo apt-get update</code></pre>
</li>
<li>Now install the bumblebee:
<pre><code class="language-bash"> sudo apt-get install bumblebee bumblebee-nvidia</code></pre>
 NOTE: If you do not want the discrete card to be enabled at all please use the following command instead:
<pre><code class="language-bash"> sudo apt-get install --no-install-recommends bumblebee</code></pre>
 This will not install the drivers for your discrete card but if you want in the future to have it just enter the following and get it installed:
<pre><code class="language-bash"> sudo apt-get install bumblebee-nvidia</code></pre>
</li>
<li>Some people recommend to reboot now although I saw the results without doing so.
</li>
<li>If you installed the nVidia driver too, in order to launch a program and use the discrete card you must use the following command:
<pre><code class="language-bash"> optirun _appnamehere_ &amp;</code></pre>
</li>
</ul>
In order to get support for 32-bit applications with nVidia you should do the following:
<pre><code class="language-bash">sudo apt-get install virtualgl-libs-ia32</code></pre>
I hope that you will find this guide helpful. I got amazing results with it myself: Power draw before: 26-28w Power draw after: 11-13w
<a href="http://bumblebee-project.org/">Bumblebee Project Website</a>

A guide that guides you through the installation of Bumblebee to manage the discrete along with the integrated graphics card.

nVidia Optimus Support in Ubuntu 12.04 LTS - How to get the most of your battery life guide


Your **hostname** or otherwise Computer Name is the name that your computer uses inside the network it is connected to in order to distinguish itself. Usually you setup your own during the setup of the Operating System but if for any reason you want to change it check out the guide below.

- Open a terminal ```CRTL + ALT + T```

- Execute:

    ```bash
    sudo vim /etc/hostname
    ```

- Change it to whatever you want your name to be, **Save** and **Close** the file

- Now we have to update our **hosts** file in order to route ourselves.

    ```bash
    sudo vim /etc/hosts
    ```

- Find the line where it says: **127.0.1.1 usernameHere** and change it to what you entered above and then **Save** and **Close** the file.

Enjoy your new Computer Name :)


Your hostname or otherwise Computer Name is the name that your computer uses inside the network it is connected to in order to distinguish itself. Usually you setup your own during the setup of the Operating System but if for any reason you want to change it check out the guide below.
<ul>
<li>Open a terminal <code>CRTL + ALT + T</code>
</li>
<li>Execute:
<pre><code class="language-bash"> sudo vim /etc/hostname</code></pre>
</li>
<li>Change it to whatever you want your name to be, Save and Close the file
</li>
<li>Now we have to update our hosts file in order to route ourselves.
<pre><code class="language-bash"> sudo vim /etc/hosts</code></pre>
</li>
<li>Find the line where it says: 127.0.1.1 usernameHere and change it to what you entered above and then Save and Close the file.
</li>
</ul>
Enjoy your new Computer Name :)