Just yesterday, I was working on some Terraform infrastructure code and doing a code review with a senior member of the team. During the review, they posed a simple question that gave me pause:

"Do you understand the difference between 'looks good' and 'is good'?"

At first, I thought this was their way of telling me that my code wasn't good enough. However, after reflecting on it for a while, I began to discern a deeper meaning in their question.

Looks good.

Making a code look good is easy.

module "label" {
  source = "./modules/label"

  tags       = var.tags
  namespace  = var.namespace
  name       = var.name
  stage      = var.stage
  delimiter  = var.delimiter
  attributes = compact(concat(var.attributes, ["cluster"]))
}

module "vpc" {
  source = "./modules/vpc/"

  tags       = local.tags
  namespace  = module.label.namespace
  name       = module.label.name
  stage      = module.label.stage
  cidr_block = var.cidr_block
}

This code may appear neat and visually appealing. Someone who stumbles upon this Terraform code might say it is beautifully written and likely copied from a template provided by a popular Terraform module provider, such as CloudPosse.

However, this does not necessarily mean the code is "good." At least, not on the inside. One could argue that the only thing that truly matters is the exported interface of the module, and the underlying complexity of the implementation may be irrelevant.

While the code may look clean and organized on the surface, the actual quality and maintainability of the codebase depend on factors beyond just its aesthetics. The complexity of the logic, the efficiency of the implementations, the robustness of the error handling, and the overall alignment with the project's requirements are all important considerations that cannot be fully assessed based on the appearances alone.

In software development, it is important to strike a balance between the visual appeal and the underlying quality of the codebase. Code that looks "beautiful" on the outside but is convoluted or difficult to understand on the inside may still pose challenges for long-term maintenance and future enhancements. The true measure of "good" code should encompass both its external presentation and its internal structure and implementation.

locals {

  defaults = {
    label_order = ["namespace", "stage", "environment", "name", "attributes"]
    delimiter   = "-"
    replacement = ""
    # The `sentinel` should match the `regex_replace_chars`, so it will be replaced with the `replacement` value
    sentinel   = "~"
    attributes = [""]
  }

  # The values provided by variables superceed the values inherited from the context

  enabled             = var.enabled
  regex_replace_chars = coalesce(var.regex_replace_chars, var.context.regex_replace_chars)

  name               = lower(replace(coalesce(var.name, var.context.name, local.defaults.sentinel), local.regex_replace_chars, local.defaults.replacement))
  namespace          = lower(replace(coalesce(var.namespace, var.context.namespace, local.defaults.sentinel), local.regex_replace_chars, local.defaults.replacement))
  environment        = lower(replace(coalesce(var.environment, var.context.environment, local.defaults.sentinel), local.regex_replace_chars, local.defaults.replacement))
  stage              = lower(replace(coalesce(var.stage, var.context.stage, local.defaults.sentinel), local.regex_replace_chars, local.defaults.replacement))
  delimiter          = coalesce(var.delimiter, var.context.delimiter, local.defaults.delimiter)
  label_order        = length(var.label_order) > 0 ? var.label_order : (length(var.context.label_order) > 0 ? var.context.label_order : local.defaults.label_order)
  additional_tag_map = merge(var.context.additional_tag_map, var.additional_tag_map)

  # Merge attributes
  attributes = compact(distinct(concat(var.attributes, var.context.attributes, local.defaults.attributes)))

 # code removed for clarity
}

This is what the label module code looks like on the inside. It's a real mess of Terraform code that would be very challenging to maintain. Although I understand the purpose of each function and local variable here, trying to make edits after being away from this codebase for a month would likely require several minutes to reorient myself and figure out the implementation.

But can this code truly be made "good"?

To improve this, we must first focus on simplifying it. Before proceeding, I'd like to reference the "Zen of Python" principle:

Simple is better than complex.
Complex is better than complicated.

The key distinction here is that complexity, while not ideal, can be justified if it is necessary to accomplish a task. Complication, on the other hand, simply means the code is disorderly and difficult to understand without necessity.

In the case of this label module, the current implementation appears to be overly complicated, rather than appropriately complex. To make this code "good," the focus should be on reducing the complication and unnecessary complexity, breaking down the logic into more modular and understandable components.

Complex vs Complicated

Complexity means that something is hard to understand for a reason, because it requires you to study the complexity, and there is no pragmatic way to avoid it. Compilers might be complex for a programmer because they accomplish a complex job. The nature of compilers is inherently hard to understand, so it cannot be reduced to simpler elements.

Complication means something is difficult to understand and disorderly. In other words, it's a mess. Although this is subjective, one could argue that Helm templates are complicated. However, complicated things can be simplified into simpler and more maintainable components using tools like Kustomize or domain-specific languages (DSLs) like Jsonnet.

The key difference is that complexity arises from the inherent nature of a system, while complication is more about the disorder and difficulty in understanding something. Complexity cannot be easily reduced, while complicated things can often be simplified through the use of appropriate tools and techniques.

Is Good

For something to become good, it must possess a good internal mechanisms and external interface. The code must be maintainable.

In the above example: The sheer number of variables and the nested conditional logic can make the code complex and potentially difficult to maintain or extend in the future, especially for larger or more complex software projects.

In summary, while the code "looks good" in terms of its structure, readability, and adherence to coding best practices, the statement "is good" requires a more in-depth evaluation of the code's actual functionality, performance, and overall quality, which cannot be determined solely from the provided code snippet. A comprehensive assessment of the code's suitability, robustness, and alignment with the software's requirements would be necessary to determine whether the code "is good" or not.

1. One resource will have less change compared to another resource. For example: your VPC resource may not change for the lifespan of the project but the application resource may change every day. How can I structure it in a way that I can make sure the changes in application don't have any relationship with the change in VPC?

2. Changes in some resources are significant. Such as IAM Roles, Accounts, Billing, etc. They may bring down the whole system. How to make sure it's properly encapsulated? So that the code responsible for the application is guaranteed to have read-only access to the IAM Roles.

Layered Approach

Then, I read this fantastic write-up I've included in the references section of this blog. We can think differently. What if we can distinguish every resource in different layers based on the two principles:

1. Rate of Change: The resources with a similar rate of change should go in the same layer.

2. Encapsulation: A layer should encapsulate all the necessary resources which change with it.

If you think of these layers, they resemble somewhat like the OSI layer model in computer networking. Each layer encapsulates the data passing only the required data to the lower layer. The rate of change and the effect of change increases drastically as you go up the layer chain.

Layer Representation

This is the diagram of different layers as presented in the blog by Lee Briggs.

We can discard the upper two layers. It's best if we manage those layers manually because the lifespan of those resources is generally equivalent to the lifespan of your whole IaC project. And, we have this sweet problem always existing of "if IaC manages state using a resource which can't be managed by that IaC, what manages that resource? For example s3 bucket". For this kind of resource, it's best to have some scripts that do it.

So, the approach that I'd like to take is the following.

If you prefer a tree view, this is how I've structured the codebase written using Pulumi SDK in Golang.

./layers/
├── application
│   ├── main.go
│   ├── Pulumi.dev.yaml
│   └── Pulumi.yaml
├── assets
│   ├── main.go
│   ├── Pulumi.dev.yaml
│   └── Pulumi.yaml
├── certs
│   ├── main.go
│   ├── Pulumi.dev.yaml
│   └── Pulumi.yaml
├── compute
│   ├── main.go
│   ├── Pulumi.dev.yaml
│   └── Pulumi.yaml
├── data
│   ├── main.go
│   ├── Pulumi.dev.yaml
│   └── Pulumi.yaml
├── ingress
│   ├── main.go
│   ├── Pulumi.dev.yaml
│   └── Pulumi.yaml
└── network
    ├── main.go
    ├── Pulumi.dev.yaml
    └── Pulumi.yaml

Complete Project Structure

If you look at the complete project structure, it resembles something like this.

.
├── assets
├── base
│   ├── Dockerfile.infras-builder
│   └── Makefile
├── env
│   └── override.mk
├── go.mod
├── go.sum
├── internal
│   └── file
│       └── main.go
├── layers
│   ├── application
│   │   ├── main.go
│   │   ├── Pulumi.dev.yaml
│   │   └── Pulumi.yaml
│   ├── assets
│   │   ├── main.go
│   │   ├── Pulumi.dev.yaml
│   │   └── Pulumi.yaml
│   ├── certs
│   │   ├── main.go
│   │   ├── Pulumi.dev.yaml
│   │   └── Pulumi.yaml
│   ├── compute
│   │   ├── main.go
│   │   ├── Pulumi.dev.yaml
│   │   └── Pulumi.yaml
│   ├── data
│   │   ├── main.go
│   │   ├── Pulumi.dev.yaml
│   │   └── Pulumi.yaml
│   ├── ingress
│   │   ├── main.go
│   │   ├── Pulumi.dev.yaml
│   │   └── Pulumi.yaml
│   └── network
│       ├── main.go
│       ├── Pulumi.dev.yaml
│       └── Pulumi.yaml
├── main.go
├── Makefile
├── pkg
│   ├── acm-certificate
│   │   └── main.go
│   ├── ecr-repository
│   │   └── main.go
│   ├── ecs-cluster
│   │   └── main.go
│   ├── ecs-service-v2
│   │   ├── main.go
│   │   └── types.go
│   ├── label
│   │   └── main.go
│   ├── load-balancer
│   │   ├── main.go
│   │   └── target_group.go
│   ├── mongo-database
│   │   └── main.go
│   ├── postgres-database
│   │   └── main.go
│   ├── s3-cloudfront-website
│   │   └── main.go
│   ├── security-group
│   │   ├── main.go
│   │   └── rule.go
│   └── ssm-parameters
│       └── main.go
├── policies
│   └── ssm-parameter.access.json
└── targets
    ├── docker.mk
    ├── go.mk
    └── pulumi.mk

Explanation

I've heavily used Makefile for this architecture.

targets

Everything that needs to be run from automation is grouped into the CLI it invokes. For example: dockerized commands, go into the docker.mk, go building/ linting/ vendoring commands go int othe go.mk file

pkg

This folder contains the Pulumi components that I've created to abstract all the resources required for one logical component. For example: the ecs-service-v2 creates EBS volumes, ECS task definitions, and ECS service itself.

layers

Every layer of the above diagram resides in a respective folder inside of the layers folder. Each of those packages produces a binary which is then passed with LAYER_NAME to the make command to operate on a single layer at a time.

env

This folder contains the environment variables required for the whole deployment to function. It also sets some handy variables like GOCACHE, GOPATH, and PULUMI_HOME to ensure I have a caching mechanism and a pretty fast build cycle in local environment.

These things will be directly set when running the IaC in CI/CD platform.

Reference

Structuring your Infrastructure as Code | lbr. (leebriggs.co.uk)

Lots of syntax doesn't work out of the box and I couldn't make it work with third-party npm modules such as jsonwebtoken etc. I couldn't even use esbuild to generate a single file build.

From the documentation of nginx:

njs is a subset of the JavaScript language that allows extending nginx functionality. njs is created in compliance with ECMAScript 5.1 (strict mode) with some ECMAScript 6 and later extensions. The compliance is still evolving.

I won't go with the installation process because you can get it easily in the documentation here. Also, you'd have to add the following repositories to have nginx packages available if you're also using Ubuntu.

# NGINX Repository
deb http://nginx.org/packages/mainline/ubuntu/ jammy nginx
deb-src http://nginx.org/packages/mainline/ubuntu/ jammy nginx

Running Hello World

After, we've installed the njs module. Let's go ahead and write the js code for straight-up firing the hello world.

@machine:/etc/nginx sudo cat > index.js << EOF
> var hello = function (r) {
    r.return(200, "Hello world!");
};
export default { hello: hello };
> EOF
root@machine:/etc/nginx

This is a very simple handler, which will return the string "Hello world!" with status code 200.

Let's plug it into the nginx.conf file.

user  nginx;
worker_processes  auto;

load_module modules/ngx_http_js_module.so;

http {
  js_import index.js;
  server {
      location / {
          js_content index.hello;
      }
  }
}

Now, when I send GET request to localhost with curl, I get the expected response.

root@machine:/etc/nginx# curl localhost
Hello world!
root@machine:/etc/nginx#

This is already very cool. But, we can do even more.

Implementing Dummy Auth and User Service

Let's create two simple services; one auth service and one for consuming the auth service.

Auth Service: Code

This is a fairly simple API. The login handler takes a POST request with username JSON key and generates a JWT token. The verify handler verifies the JWT token passed as a bearer token.

Let's test our auth service if it's working as intended.

regmicmahesh@machine:/etc/nginx$ curl -s localhost:8080/login -XPOST -d '{"username":"mahesh"}'
{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTUyMjUzMjMsInVzZXJuYW1lIjoibWFoZXNoIn0.y7kOx9dScMwg0XjRj1AAa9xh3rZp0GzEzR0FC1f3x-w"}
regmicmahesh@machine:/etc/nginx$ curl -s localhost:8080/login -XPOST -d '{"username":"mahesh"}' | jq -r ".token"
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTUyMjUzMjYsInVzZXJuYW1lIjoibWFoZXNoIn0.XXkLCwspeYYy8kCONSSPve1T7AYX94GgzNB1rrdOygk
regmicmahesh@machine:/etc/nginx$ k^C
regmicmahesh@machine:/etc/nginx$ curl localhost:8080/verify -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTUyMjI5NTksInVzZXJuYW1lIjoibWFoZXNoIn0.GXnnmLuXRr8tu1Sx08DXcgs7N6au4huku-S62sR0JCU'
{"exp":1695222959,"username":"mahesh"}

User Service: Code

The user-service simply prints out the value of the header X-User-Id .

Now, we need to configure our nginx such that, every request first goes through the auth service, decodes the token if possible, and passes the username to user-service

Writing Nginx Config

First of all, let's configure the /verify .

 location = /verify {
   internal;
   proxy_pass http://localhost:8080/verify;
}

This is okay enough. The auth-service is running on port 8080. This should pass the request to that endpoint whenever requested.

Now, we can simply use auth_request /verify . But, where's the fun in that? We want to set the username in the header dynamically.

Let's write a simple njs script to send a request to this endpoint.

function verify(originalR) {
  function callback(r) {
    originalR.return(r.status);
  }

  originalR.subrequest("/verify", { method: "GET", body: "" }, callback);
}
export default { verify };

The NJS script above simply sends the request to /verify but doesn't send the request body. It sends only the headers.

Let's add one more block in nginx to use this js function.

 location = /auth {
   internal;
   js_content index.verify;
}

Now, we need to use this /auth endpoint as the authentication endpoint and everything should be good. We just wrapped our actual verification endpoint with a proxy. Why? Because now we can add our JS logic to set the header.

Let's define a variable to hold the value of the username.

server {
    set $username "";
# other code
}

Now, we can set this variable from the NJS script.

function verify(originalR) {
  function callback(r) {
    if (r.status === 200) {
      const body = JSON.parse(r.responseText);
      originalR.variables.username = body.username; 
      originalR.return(200);
    } else {
      originalR.return(401);
    }
  }

  originalR.subrequest("/verify", { method: "GET", body: "" }, callback);
}

export default { verify };

Now, we can send this variable as a header to the actual user service.

Adding it in the location, block it comes as follows:

location / {
    auth_request /auth;
    proxy_set_header "X-User-Id" $username;
    proxy_pass http://localhost:8081;
}

Let's take it for a test-run. It works!

regmicmahesh@machine:/etc/nginx$ curl  localhost -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTUyMjI5NTksInVzZXJuYW1lIjoibWFoZXNoIn0.GXnnmLuXRr8tu1Sx08DXcgs7N6au4huku-S62sR0JCU"
Request Received From: mahesh
regmicmahesh@machine:/etc/nginx$

Thanks for reading my blog. I don't think this is the best way, but just wanted to share what's possible with NJS scripting.

Let me start this blog with how it started, I wanted to make a super simple pod troubleshooter to provide for developers. Developers were having a hard time running migrations, checking application logs, and everything. I thought, okay, I'll create a simple tool which will do one simple thing; provide live tty session inside a running pod.

Although I used Weavescope for the project need, I thought it was always exciting to learn how these tools work and how can I develop one myself. So, I decided I'll build one live tty API for the pod myself.

REST API of Kube-API Server

Let me give you a little bit of background on what is this kube-apiserver thing.

Kube-API Server is one of the major services in the control plane which provides a REST API to change/configure the cluster's state. All the things (kubelets, controller manager) need to communicate with the API Server to perform any change in the cluster.

Now, it was clear. If I want a shell in the pod, I need to go through the kube-api server. But how? I wanted the REST API documentation or any reference on how can I send requests to the kube-API server.

Let me teach you a nice hack to find out what requests are you making when you're using kubectl. You can change the verbosity to level 10 to get the exact curl request you're making with the API Server.

kubectl get pods --v=10
...
curl -v -XGET -H "User-Agent: kubectl/v1.25.4 (linux/amd64) kubernetes/872a965" \
 'https://[redacted].com/api/v1/namespaces/default/pods?limit=500'
...

It's that simple to understand what request is kubectl making to fetch the list of pods in your cluster.

You can also get the OpenAPI schema of the REST API of Kubernetes.

curl --insecure -L "https://$API_URL/openapi/v2" \
    -H "Authorization: Bearer $TOKEN"

This will provide you with the complete open API schema of the kube-api server. You can then visualize this with tools like redoc, swagger, etc., and get a proper hold of the API.

REST Endpoint for pod/exec

Our goal is to extract the endpoint of exec-ing inside of the pod. You can either get it using the kubectl as I showed you or just explore through the schema. Either way, the endpoint is this.

${this.config.url}/api/v1/namespaces/${namespace}/pods/${podName}/exec?command=${cmd}&stdin=true&stdout=true&stderr=true&tty=true

But the catch is, that this endpoint doesn't work on HTTP, it upgrades your request to WebSockets. You can't do something like okay, I'll fire up an HTTP request with the command I wanna execute and show the stdout. The response to this endpoint is always a WebSocket upgrade, and the stdout/stderr is sent as a WebSocket message.

curl -XPOST -H "X-Stream-Protocol-Version: v4.channel.k8s.io" \
 -H "X-Stream-Protocol-Version: v3.channel.k8s.io" \
 -H "X-Stream-Protocol-Version: v2.channel.k8s.io" \
 -H "X-Stream-Protocol-Version: channel.k8s.io" \
 -H "User-Agent: kubectl/v1.25.4 (linux/amd64) kubernetes/872a965" \
 "https://$API_URL/api/v1/namespaces/default/pods/nginx-deployment-cbdccf466-qjfvw/exec?command=python&container=nginx&stdin=true&stdout=true&tty=true"

The response will contain the header Connection: Upgrade and Upgrade: SPDY/3.1 This means the connection is upgraded to WebSocket.

curl won't be able to take you long enough with WebSockets, we can use another tool similar to curl wscurl to send WebSockets requests with curl-like API.

If you're using curl, make sure to specify the subprotocol (i.e. X-Stream-Protocol-Version) with the flag -s .

wscat -H "Authorization: Bearer $TOKEN" \
   -c "$URL" \
   -s "v4.channel.k8s.io"

Message Format from Kube-API Server.

The message you receive from kube-api server over WebSockets is in a specific format. The first byte contains the messageType and the remaining bytes are the actual message from the TTY.

There can be three types of messages: stdin, stdout, stderr .

In this way, you need to filter out the messages whether they're in stdin, stdout, or stderr.

For example: if you establish a connection with websocket, wscat will print something like this:

You see the first byte in the message received from the server. It's 0x01 because it's sent in stdout by the pod when you open a shell. And, your terminal doesn't know how to render that.

Will continue in Part 2.

Throughout my career, I've worked with numerous frontend frameworks, backend frameworks and operational strategies and technologies. This brings back a lot of painful and happy memories. In this blog, I've written about my views from my experience which may or may not be correct.

One problem I've always faced in the majority of the projects is as the projects grow it gets increasingly difficult to change things, observe things and gain a better understanding of the system. When we aren't able to have a good sense of the system, we can't properly architect the new change.

One recent example; I was leading in moving the auth-module of a system from self-backend implementation to Google provider. When I started the project it was really simple, I had implemented the auth module as a separate testable entity that could be plugged into other modules and easily modified. As the project grew with other developers, they started plugging in middlewares, modifying attributes of the auth module and injecting unneeded things to the auth state just because it was properly designed to hold the state in the request-response lifecycle.

Now, I thought to change the internal implementation of the auth module, and sweet hell Mary, everything was a mess. Our middleware broke, and authorization didn't work. Most of the modules that depended on the auth module broke.

Another example; I was leading the development of a data labeling system. We envisioned this system would be complex, so we chose to separate systems into 3 different components. This time, I had bootstrapped the projects and the code was in proper shape in the initial commit. But, soon after from Day 2, there were blockers. Separate components had separate developers working on them. The interaction between components was over HTTP, and communication issues between these components ate up a lot of the development time. By a lot, I mean the development time was extended significantly. Now, when I think about it, it boils down to "premature optimization is the root of all evil".

Another example; This was a streaming site built with a claimed "Super Fast" NodeJS framework at the time. I thought, okay streaming is a tough job. Let's write this service with NodeJS and a custom ffmpeg wrapper in NodeJS. The super-fast framework was so slow to develop, that it had no community plugins. I wrote a lot of things myself which I could have got for free. If I'd sacrificed 1% speed and used a slower framework, my productivity would be 100x.

What did I learn from it?

I learned two key things from it:

1. Developers often fail to understand the intention behind the code.

2. You don't need to think about scaling to 10 million when you're writing your first line of code.

3. There is no such thing as "It worked for X corp, it should surely work for us."

4. Don't go with "Yeah, I've worked with X framework between these choices, and I think it's a good fit."

5. Performance isn't always the top priority.

How do I do it now?

Now, I've seen enough failures to identify the correct one. I'm confident enough to tell that whatever you're building, it should follow these characteristics:

• should have as little magic as it can.

• dx is not a thing to ignore.

• Most of the things should be automated.

• Interfaces and boundaries should be thought of at first.

• The system should be observable.

• There should be a proper document in place to understand "why" for all ADRs (architecture decision records)

In the starting, I'll start with some BaaS (Firebase, Supabase, Strapi maybe) because it's not worth investing a lot of code and developer time when your (potential) clients and software don't even understand each other.

Most of the BaaS makes things observable, you can know which services are getting most requests, which queries are taking the most time, what time you are getting a lot of traffic, which is the bottleneck in your system etc. You also get sweet other things out of the box such as authentication/authorization, and data persistence layers and the best of them all is dx with it.

After a certain period, you'll get a perfect understanding of which way your software is heading. You can slowly start separating the bottlenecks of the BaaS platform into separate services. You write your code this time, as serverless functions if possible for your service. And you plug it into your BaaS.

Now, you need to start finding out how that BaaS does certain things and how'd you on about doing that on your own, research on the database performance, language, toolings etc.

After a certain level of maturity, you'll realize your business logic is getting too complicated to tune your BaaS and you need to do it on your own. This is the moment you start slowly rewriting your backend and shifting your API into your backend, one service at a time.

At last, this is my personal experience. Let me know if you have any conflicting views or I misunderstood something, I'm learning every day and would be very happy to correct myself.

Hardening refers to decrease the potential of a system getting exploited by reducing the attack surface. It may range from adhering different policies like Least Privilege, Zero Trust and also contain different steps like automatic security updates, firewalls, rotating passwords etc.

For hardening a system, basically the system-ops should cut-off unnecessary provisioned processes, services and vast majority of other resources currently used by the system unless they're bare minimum or essential for the system to function.

There are a million steps to be taken in order to prepare an absolutely stone hardened system but we'll look at some essential steps in hardening a docker image so that next time someone tries to own your docker container, they'll have hard time.

Limit the build data with .dockerignore or unnecessary copy.

Most of the times when building a docker image, it's a wild practice to move a directory directly inside the image, but this increases the attack surface as .env files, configuration folders, log files may be moved inside the docker image which gives the attacker more information already present inside the docker container regarding the system.

Even if the data are too many to be copied without doing a wild-card copy, I'd suggest to use a .dockerignore file which behaves similar to .gitignore and it stops the files from being copied which are mentioned there.

Use smaller images or scratch if possible.

Using a larger docker bases which consist of a thousand of tools your application doesn't need to function but increase the attack surface as there's chances of them having security issues is a great way of helping the hacker.

Try to use a very small image. Also, don't make a nightmare for developer by using images like alpine when the application is supposed to use very complicated libraries as alpine builds everything using musl and sometimes it's a nightmare.

My preferred docker images are slim builds of debian. Also, it's a lot easier when you're using statically compiled languages like go, which can run natively inside docker container when you just ship a container with only the binary. Think about the attack surface reduction when everything the filesystem contains is a single binary.

Downgrade to non-privileged User.

Getting rid of the root user in a docker image is just a two line configuration change. By doing just this simple change, we introduced a whole new problem of privilege escalation to the potential attacker.

Mount host file system as read-only.

Often times, we need to mount host data for reading configuration, or many other reasons for the docker container to function properly. For this, if we mount the host file system with full RW access chances are the potential attacker will misbehave with the mounted file system. To mitigate to some extent, we should mount the file system in read-only mode. We should reduce the access in the host file system by stripping down to the only required folder / file.

Don't expose unnecessary ports.

In development environment, it's a good idea to expose multiple ports, as the developer may require to run remote debugging in the container, or an extra development only network access for whatever reason but those ports should be strictly shut down while running with live users. It's a good idea to expose only the port listened by the main process running inside the docker container.

Never run in privileged mode

This requires no explanation. Running a container in privileged mode means it can do basically anything the host system. This is the worst thing through which you not only risk the container but also risk the host system.

Other Extra Suggestions

Enable Scan on Push in ECR.

Elastic Container Registry (ECR) service provided by Amazon can scan for security issues in your docker images as soon as you push them. This enables you to know about the potential threats in the docker image and quickly update / patch the packages which have CVEs assigned to them.

The image below correctly explains the three main pillars provided in javascript to support functional programming paradigm.

Map is used to create a new array based on certain manipulation of every item of existing array but not removing them.

Filter is used to create a new array based on certain condition of existing array.

Reduce is the master of all which includes a whole pipeline and can do map, filter and much more. But, as you see in the diagram it is used to produce a single value at the end.

Before getting hands-on with reduce, we need to understand few concepts first.

Accumulator

Accumulator is used to store the current value of our reduce pipeline. Initially, if we provide any initial value to the reduce function, accumulator equals that value if not accumulator takes the first value in the array and the iteration starts from second item. Not clear? Look at the examples below.

Let's start with one of the most basic reducing operation.

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr.reduce((acc, el) => acc + el);
console.log(num); // 55

Here, the reduce function takes 1 as initial value ( first item in the array ) because no initial value is provided. After taking the first item as initial value, the invocation of function starts with the second item in the array.

Initially, the accumulator stores the same value as the initial value.

So, the initial value is 1, the accumulator holds the value 1, and the current element is 2.

After running the function we passed inside the callback, the returned value of our function is the new value to be set in the accumulator.

After one iteration.

So, the accumulator holds to value 3, and the current element is 3.

After second iteration.

So, the accumulator holds the value 6, and the current element in 4.

So, the accumulator is responsible for holding the returned new item after every iteration which is again passed while invoking the function for next item.

Guess, the above example makes it crystal clear now.

Now, let's try to do the same thing but providing initial value.

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr.reduce((acc, el) => acc + el, 5);
console.log(num); //60

Here, we pass another argument to the reduce method of array 5. Now, the reduce method will take 5 as initial value. The iteration begins on first item because it's not taken as initial value.

So, the initial value is 5, the accumulator holds the value 5, and the current element is 1.

After one iteration.

So, the accumulator holds to value 6, and the current element is 2.

After second iteration.

So, the accumulator holds the value 8, and the current element in 3.

Hope you understood the process.

Now, let's understand the whole syntax.

reduce((acc, el) => { ... } )
reduce((acc, el, i) => { ... } )
reduce((acc, el, i, arr) => { ... } )

We tried the first syntax already. The first item we pass is the accumulated value which is equal to the initialValue ( if we pass ) or the first item of array initially.

The second syntax is also near-equivalent but it takes a third argument i which is equal to the current index of the array.

The third syntax passes the whole array as first argument.

Keep in mind, only the first two are required. All other are optional parameters, pass it if you need them.

Now, you've a foundational knowledge of reduce, let's try to see how it's a swiss knife of functional programming.

Converting an array of number to array of strings.

For this operation map may be suitable and if you do it via map, you can do something like this...

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr.map(el => el.toString())
console.log(num); // ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10"]

This is a lot simpler approach. Let's try to approach this with reduce.

First let's think how we can solve this like a functional programmer.

The initial value for this operation should be blank array i.e. []. Now, on every reduce operation we convert the number into string and push into the array.

Pretty straight-forward right?

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr.reduce((acc, el) => [...acc, el.toString()], []);
console.log(num);

If you're confused with the part [...acc, el.toString()], it means to spread the value of current acc array in the place of ...acc which means if acc equals to [1,2,3] then our new array will be replaced as [...acc] => [1,2,3], here the value inside of acc array is spread in-place.

After the current value held by the acc, we are pushing our current el.toString().

The flow can be explained as...

At first, acc = [] and current element = 1

At second, acc = ["1"] and current element = 2

At third, acc = ["1" , "2"] and current element = 3

Pretty straight forward right? If you're having issues with spread operator, consider this.

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

const num = arr.reduce((acc, el) => {
 acc.push(el.toString());
 return acc;
}, []);

console.log(num);

Now, what if only even number is required.

Let's try with filter and map.

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr
    .filter(el => el % 2 == 0)
    .map(el => el.toString())
console.log(num);

This does the job. Now, let's try implementing this with reduce.

So, the initial value will be []. Our callback function should return the accumulator if element is odd, else add the stringified-element to the accumulator and return it.

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr.reduce((acc, el) => {
  if(el % 2 == 0) acc.push(el.toString());
 return acc;
}, []);
console.log(num);

Now, this is getting interesting, right?

Let's try to implement a reduce function which reverses an array. It's quite simple because we also have reduce right, which reduces the array from left.

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr.reduceRight((acc, el) => {
  if(el % 2 == 0) acc.push(el.toString());
 return acc;
}, []);
console.log(num);

We can also do this though...

const arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const num = arr
    .filter(el => el % 2 == 0)
    .map(el => el.toString())
    .reverse()
console.log(num);

This is just the surface of the power of reduce. You can do much more based on your needs. If you've more examples, feel free to drop those in comments.

Further Reading: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/Reduce

Luckily, it's pretty straight forward to remove that change from git history both from your local repository and remote repository.

For the demonstration, I've create a local git repository.

And populated with some fake git commits to demonstrate what you may actually do.

$ git log --oneline
d5d9f70 (HEAD -> main) some more changes to main
c05631b Added Password
ac547a8 Added Main File to Project

Editing Commit

Here, I have made clear using commit message to show that the commit with has c05631b adds sensitive information to our git history. Now, we need to remove that.

If you're having hard time finding which commit actually changed the password. You can use the following git command...

git log -p fileName

Now, run git rebase using this commit hash to rebase our git history from that commit onwards.

$ git rebase -i c05631b
pick c05631b Added Password
pick d5d9f70 some more changes to main

# Rebase ac547a8..d5d9f70 onto ac547a8 (2 commands)
# <Hidden for Content Purpose>

Now, edit the text in the editor. We need to edit the commit which added password into the git history. So, I'll rewrite the first line as follows...

edit c05631b Added Password
pick d5d9f70 some more changes to main

# Rebase ac547a8..d5d9f70 onto ac547a8 (2 commands)

Note the first line, I've replaced the pick with edit. Now, after saving and exiting the text editor, git will put me right into the staging area of that commit.

Now, edit your password file. I will remove the password from .env file in this case.

And, I will add the changes into the commit using the given command.

$ git commit --amend -a -m "Removed Sensitive Content"

This will update the message in that commit and also the changed file.

Now, I will continue the rebase using the following command...

$ git rebase --continue

Now, git will try and update the git history by updating that specific commit with our new change.. We may also get some conflicts in the process.

Dropping Commit

This section is optional. If that's all you want you can go ahead and skip this section and follow the pushing into GitHub section.

Now, I will drop the same repository.

Use the git rebase command as before, but keep in mind the commit hash will be changed. So, again grab your commit hash using git log.

git rebase -i ab5edff

Now, change the pick line into drop which will actually remove the commit from git history.

Make sure you aren't dropping a repository which has other changes, it's generally bad idea to drop a commit. Or, it was a bad idea to create :D.

$ git rebase -i c05631b
drop c05631b Removed Secure Content
pick d5d9f70 some more changes to main

# Rebase ac547a8..d5d9f70 onto ac547a8 (2 commands)
# <Hidden for Content Purpose>

Now, after saving and exiting this file. git automatically removes your commit and updates the git history.

$ git log --oneline
9f865e5 (HEAD -> main) some more changes to main
ac547a8 Added Main File to Project

YAY!

Pushing To Github

If this repository is already pushed into the github, it's always a bad news to rebase a remote repository, because it'll make conflict on all of your co-workers git history.

The command given below will do the job.

$ git push origin main --force

Two of the major concerns we had while designing the system was efficiently converting the video into HLS format, so that user can have better experience and the video adapts with change of the bandwidth, other concern is regarding tracking the watch time and recommending other videos to the user.

The second one is a lot easier because of prior experience of developers in our team with recommendation system on python and websockets for tracking everything the user does.

For converting the video into HLS format, we had a major hassle as this requires a lot of domain knowledge about video encoding and conversions.

Luckily, ffmpeg acted as a saviour for all of our issues. We could have used node-ffmpeg but we went with native ffmpeg in order to keep it simple, as we’re going distributed we may also have an extra endpoint layer of ffmpeg in aws to handle the video conversion in and out from AWS S3.

$ ffmpeg -i filename.mp4 -codec: copy -start_number 0 -hls_time 10 -hls_list_size 0 -f hls filename.m3u8

We used this script in order to convert a file from mp4 format to HLS.

Currently, we have a bit longer script which re-encodes the video into multiple resolutions for seamless and lag-free streaming experience.

In order to let this script to be executed, we needed a message broker which takes incoming video and consumes it to convert it to HLS.

For that problem, we used bull to act as message broker.

We created a queue at first called VideoQueue which will route our messages into the consumer.

Then, we wrote a very simple consumer function which just encodes the raw video file. Keep in mind, if you’re planning to do the same, you must validate all the inputs. Don’t throw native packages randomly as the file may not be what you want and unsanitized may lead to RCE.

This is our internal video converter which uses the shell script in order to convert the file and save it into movies folder.

Bull is extremely easy to use and configure.

It just stores your message into redis and consumer pulls from it. We’ve had a pretty good response time and conversion status with it as of now. We’re currently experimenting other video conversion formats to serve best traffic under certain video bandwidth.

Thanks. Too busy these days. No time to write articles.

Note: This is not an introduction to JWT ( JSON Web Tokens) or any tutorial to implement JWT from scratch.

In JWT, the token is the most important piece of information to identify the user in the system. If user is able to tamper, see and modify the JWT by any means, then he/she’ll be able to perform identity theft with XSS / CSRF attacks when not secured properly.

Let’s try to approach this problem by whatever comes in mind.

What if we tried to save the token in local storage and send it in header on every request?

There are many things wrong with this approach, One of the many being user has complete access to localstorage CRUD operations in the browser with few lines of javascript. This is extremely vulnerable to XSS attacks.

If a potential attacker has access to local storage, he can perform whatever he desires while the backend recognizes him to be the JWT Stored User.

And he hacker will have access forever, now what if user changes password what will you do?

Store it in database.

If you store the credentials in database and validate if the password has changed since the last token was issued, it’ll help you a little bit. But that’s not the point of JWT. The point of using Token Based Authentication like JWT is make sure the authentication system is maximum stateless as possible.

Let’s change the thought a little bit. What if the token expires after short time? And a new token is issued.

What ? A user logs in every(short)time?

No, let’s not do that way. It’s a terrible UX to login everytime. You can minimize UX such a terrible way just to make sure hacker doesn’t get access for long time.

Let’s use another token called refresh token which will stay with access token but have a greater expiry time. This works, now how will you make sure refresh token is used to refresh after access token is done (expired)?

Expose two endpoints

This works. But not for a long. A sensible hacker will request jwt forever and try to maximize his session as possible.

Keep in mind, our storage of JWT problem is not solved yet.

Covering the expiry part…

When you specify expiry time in jwt token, it won’t be valid token after the time has expired.

Now, the UX won’t be that bad and the token will keep on refreshing every time but how to make sure client / hacker doesn’t have reach to that token?

Using Cookies

What kind of cookies? If you store JWT in plain cookies, that’s how 90% of XSS attacks escalate to massive impact.

Using HTTPOnly Cookies

HTTP Only cookies are such kind of cookies which can’t be accessed and read from the client but will be stored on the client and passed on every request, which means the cookie works sort of like session but in a completely stateless manner.

The above code issues a two token which will be expired after 20m and 7days while the cookies can’t be read from the client.

This protects us from maximum attacks.

Little Help to Developers

This is how we can write a middleware to make sure, the token is validated and try to refresh the token if there’s any.

Make sure, your CORS enables frontend to work with HTTPOnly Cookies by following…

Keep in mind, origin can’t be wildcard when credentials is true..

You can login in this way…

I was having this problem since few months and I guess most of you are having the same issues.

Do you do development of multiple languages on your laptop and have trouble managing all the environments and dependencies of the projects?

This is one of the biggest burden in my current development environment, specially with slow HDD read/write speed.

Just now, I had issues installing the graphics library files for my development environment as Pillow in python used another version and NodeJS used another version. Updating any of them would break another.

It would be lot better, if I could isolate my programming environment but I can’t run a whole virtual machine just to have a isolation.

I mean, I don’t need a new OS, I just want to isolate my development programs, tools and codebases. Another benefit it comes is that if you stop working in any of the projects in golang, and you can remove the whole development environment with just some commands, and you’re good to go.

Isn’t that already convincing enough to try containerizing your development workflow?

Container

A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.

What a container does is, it comes as a software but contains everything it depends upon. Like if a script requires a example.so linker files and a whole g++ installed to run, a container can include all of those and still remain lightweight than you expect!

This is all the magic of linux namespaces and cgroups. The concept of isolation of processes was thought long back then and kept in linux.

So, if you’re running a virtual machine, then it runs on top of a hypervisor such as virtualbox, hyper-v, vmware, but docker containers run on the same host, but do not share the same PID list and are isolated from each other.

Keep in mind, docker container are also built secure and has minimal security concerns to think of if you configure it properly.

So, docker is just a platform which is used to control the containers effectively.

Or, quoting from the wiki:

Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers.

You can research further about docker and containers.

So, when you convert your whole development environment inside a docker container, it’s totally isolated from the host operating system and also can be managed as a whole from the host operating system, or via docker cli.

Solution

I won’t cover installing docker in this article, but you can look it up here. Installing Docker :https://docs.docker.com/get-docker/

So, in order to convert your visual studio code setup into docker container. the first you need is to choose an docker image to run.

Keep in mind, docker images are the whole software shipped with dependencies which can run isolatedly forming containers.

The most common one to use is ubuntu (if you prefer a big community) or archlinux (if you prefer lightweight)

#if you want to run ubuntu, execute this after installing docker.
docker pull ubuntu:20.04
#if you want to run arch, execute this after installing docker.
docker pull archlinux

After you’ve pulled the images for your development environment, we’ll use a volume so it’ll be easier for you to share files, to and from the container.

#For ubuntu,
docker run -d -it --name ubuntu-dev -v ~/my_files:/my_files ubuntu:20.04 /bin/sh # or /bin/bash

#For arch,
docker run -d -it --name arch-dev -v ~/my_files:/my_files archlinux /bin/sh # or /bin/bash

Now, you can check if your container is running using docker container ps

CONTAINER ID   IMAGE          COMMAND       CREATED          STATUS          PORTS     NAMES
84bc37ed215b   ubuntu:20.04   "/bin/bash"   54 minutes ago   Up 44 minutes             ubuntu-machine

The output should be similar to above one.

Now, you can open vscode and install remote pack extensions.

Install this extension pack, if you don’t have already which will allow you to setup vscode remotely in a container.

Now, you’ll see your container listed here. Just click on it and vscode will prepare a development environment in the container and make it ready to start coding.

Now, you can install all your compilers, interpreters and everything using the provided bash via package managers.

When you need to remove the development environment just remove the container and you’re good to go :)

How many times you have come across functions which automatically convert your values to required types in runtime?

For e.g. In Django whenever you define your types like this in route definitions the URL parameters are automatically parsed inside of your function.

urlpatterns = [

path("/<str:slug>", someRandomView, name="home"),
path("/<int:pk>", someRandomProfile, name="home_profile")

]

So whenever you define your views, your data types are automatically converted to the required data type.

def someRandomView(slug):
  ...

def someRandomProfile(pk):
  ...

This offers a huge convenience in your application by automatically converting your type as per your function or something hints the function wants to achieve.

Similar behavior is achieved in fast API.

Let’s define a simple function.

def getAge(age, name):
    print( f" {age} : {type(age)} , {name} : {type(name)} " )

You can see our types are nowhere defined in this piece of code. But, we can guess the user wants to age as an int variable and name as an str variable ( most probably). But python doesn’t work that way. You can call that function with any variable you want.

It would be a lot better if your code automatically gave the developer some idea about data types.

Let’s refactor that piece of code a little bit.

def getAge(age: int, name: str):
    print( f" {age} : {type(age)} , {name} : {type(name)} " )

Now, this code is a lot cleaner than your previous code. But, we still have the same problem but it aids a little bit. Your text editor may flag out wrong parameter types if you’re using some linters or running on mypy.

But, what we wanted to achieve is to make automatic type conversions in runtime. So, whenever you call the function, your arguments are automatically parsed to be converted into required data types.

Let’s try writing a decorator to print what exactly we can extract out of the function definition.

from typing import Callable

def decorator(func: Callable):

    def inner_func(*args, **kwargs):
        print(kwargs, func.__annotations__)
        func(*args, **kwargs)

    return inner_func


@decorator
def getAge(age: int, name: str):
    print( f" {age} : {type(age)} , {name} : {type(name)} " )

When we call the function using getAge(age=12,name=”Mahesh”), this is the output we get.

kwargs: {'age': 12, 'name': 'mahesh'}
__annotations__ : {'age': <class 'int'>, 'name': <class 'str'>}

Now, this makes sense. Your annotations are extracted from your function in the compile time of the function and not changed by your arguments.

So, to achieve automatic type conversion on the function, we can write all the code to convert types as mentioned in the function annotations.

#/bin/env python
from typing import Callable
def decorator(function: Callable):

    def inner_function(*args, **kwargs):

        newKwargs = {}

        for argName, typeName in function.__annotations__.items():
            if typeName is int:
                intVar = int(kwargs[argName])
                newKwargs[argName] = intVar
            elif typeName is str:
                strVar = str(kwargs[argName])
                newKwargs[argName] = strVar
        function(*args, **newKwargs)

    return inner_function


@decorator
def getAge(age: int, name: str):
    print( f" {age} : {type(age)} , {name} : {type(name)} " )

And boom, this works!

Whenever you call the function, it gets automatically converted to the required type and if it fails, it will throw the value error.