Actionhero Web Response Compression

Our company's recent internal tool was basically a database API with basic CRUD actions. There was some business logic to make sure things worked correctly, and I still like actionhero for my NodeJS APIs. The bulk of the project was in the web app (built with Quasar on top of Vue). I also wanted to try Docker and AWS Fargate, the instances are running actionhero directly (no nginx proxy).

Recently the company created a "big project". Most previous projects had 5-10 inner data hierarchies. This one had 206. The "/getFullProject" action had a real-world stress test! I thought VueJS would have issues, as in the past too many rendered objects or large arrays would slow down reactivity, but it works fine. Yay, version 3 : )

The hangup was the JSON download size. 120MB+ and it took over a minute. Too slow. Options are larger network tiers for Fargate, lazy-loading those data hierarchies from the web (API updates), or use nginx as a web proxy to gzip everything. However, I wanted a quicker solution and attempted to compress the JSON payloads directly from actionhero.

The compression project is meant for ExpressJS, but could we make it work in actionhero? Yes. Yes we can. (kind of : )

The result was the middleware above, that I only applied to a single action. My "/getFullProject" action. The `compression` project modified the raw request and response of actionhero (which are the base NodeJS http/https library). I had to turn off actionhero's normal response (data.toRender = false), just like building a file buffer or similar. Also the Content-Type header needed set before trying to compress; that is specific to the compression library (yay OpenSourceSoftware).  There are some issues with error handling, as modifying the `rawConnection` messes with how actionhero processes responses. And probably some other issues to be found! But it's an example of applying ExpressJS middleware inside actionhero.

For me, this reduced the 120MB+ payload down to under 8MB, and time from 1min+ to ~20s. All node changes, no API or infrastructure. 

It's still a slow action. I might change Docker to using a nginx proxy, and then lazy-load if needed. But for now, this experiment was sufficient and successful. 

Going with the Flow Interview Questions

I have done interviews that asked about what work I have done. Plus, of course, the overly technical interview. Two interviews really stood out as "good interviews" to me.  I actually enjoyed the interview questions! But the interviewer can tell a lot about how a person thinks and what they have really done. 

Both followed the same idea of a "going with the flow" style of interview question. Basically, the opening setting leads to a line of questioning that
1) is never wrong or discouraged
2) goes on for a while
3) requires the interviewer to think on their feet.

The point is dialog. How does the interviewee handle problem solving? Do they understand the concepts at large? The smaller parts too? Can they think on their feet? Have they done this before? And sometimes, do they give up on a challenge?

The two examples, paraphrased and reduced, went something like this:

Example 1:
Imagine a web page is up and a lawyer calls you at 3am saying it legally has to come down now. What next?
Me: Who is this? Are they for real?
Them: Always good to check! Yes, they are real.
Me: OK, I'd go turn off the server.
Them: It's on the cloud.
Me: Can I do it remotely?
Them: You could, but IT security just changed the console's access keys at midnight.
Me: Hmm, let's start at the front. I'd move the DNS to a different server.
Them: Good, but lawyer can't wait the hour for DNS propagation.
Me: Tune down at the load balancer / auto-scaling level so the server is turned off?
Them: Well, that could work, but your rules would apply to the other applications on the load.
Me: I might need to phone a friend. Who else is on my team?
Them: Teamwork, that's good. But they're all at a conference out of country, it's just you.
Them: Want to stop?
Me: Nah, let's keep trying. Can I SSH into the machine?
Them: Yes
....
Them: You don't have root access.
....

And on an on. The situation was not real, but it was real enough. The answers I gave might have worked or might not - I was never told. Their reasons for "not working" were never ending, and maybe did not make complete technical sense, but the exercise showed how far could I go? And many of these things are known from experience, not just reading blogs or doing starter projects.  The interviewer's final step when I ran out of ideas was to write a script to fill memory on the machine to starve the web server from running - I learned something! (. . . which, ironically, I ran into in our production environment a few years later - having no memory breaks stuff!)

Example 2:

Them: "We have a document processing system. Look at this chart. A goes into B, produces output 1, stored in C. D processes 1 into 2. E takes query criteria and runs against outputs 1 and 2.  We want to add process F, where would you start?"
Me: Let's add at new task at C.

Them: Cool. What technology would you use? 

Me: Looks like Redis is in A, and Redis is a good fit becase XYZ . . . 

Them: Alright, that task is working. What next?

Me: . . . .

Them: Aw, great idea! (even though it would never work. . . )

This one was more abstract because I forgot the details, and the details were really tied to their company use case. I could see the relation from the homework assignment I was assigned. Did I understand their company? Did I understand the tech being used? Was I a true "senior dev"? Every "idea" I had was a good one - or at least that's how the exercise was run. There was no need to go into the weeds, just "great idea, now what about this?!!" The interviewers were smooth too, so working with them sounded fun. They learned about me, I learned about them - win-win!

Wrap Up

This was a cool interview style that I'll keep in mind. For positions that require a lot of knowledge or experience, but you don't want to assign "build a full product" as homework, this type of questioning could be useful. Since it is very interactive, it can also pull out some personality that a coding exercise cannot.  It does require the interviewer to think on their feet and be knowledgeable of the context! Trying to fleece the interviewee could really backfire.