Monday, March 30, 2020

The Importance of Taking Time Off

Ever since I've enrolled in the Open Source Development classes at Seneca, I've had a blast. I learned about using all sorts of new technologies, got to collaborate with people who are much more skilled at programming than I am, and I've had the chance to contribute to projects that seemed interesting. I could've graduated last semester, however the project and the idea of being able to learn and ship a product under the guidance of a very experienced professor convinced me to stay for another semester just for this course.

This isn't a post to say I regret my decision, far from it. The previous week prior to writing this post, I've been waking up at 8/9 in the morning and have been working on issues all the way til usually late in the morning of the next day (seems like this isn't unique for our class). But I started to feel kind of burnt out, the things I enjoyed doing just a week ago, I started to procrastinate on or not look forward to. I decided on a simple solution, take the weekend off and just enjoy time with things not related to Telescope. Do some exercise, go for a walk(I have no idea how advisable this is currently), spend time with the family or watch a movie. 

I think it helped. As I'm writing this blog, I am content with my routine of checking slack, opening up my laptop that hasn't been opened for a a few days, browsing through outstanding issues on Github, typing the commands 'docker-compose up elasticsearch redis' then 'npm start' and fixing currently stale PRs.

I think this is applicable to probably anything and not just my situation, if you're starting to not enjoy something, take a bit of time off, enjoy other things and then re-evaluate.

Sunday, March 22, 2020

OSD700 Release 0.8

This release was almost like 0.7, three weeks(kind of) to work on it. I worked on tackling most of the existing issues assigned to me as I had an outstanding ~15 issues that didn't have a PR yet.

All links are to their pull requests.

Issue 538: Expose Search Endpoint Via Web API
A throwback to working with Express.js, a previous PR went in to include ElasticSearch, we can currently use its client on port 9200, but we didn't really integrate it with Telescope, this PR was to fix that. I created a new endpoint now at https://dev.telesope.cdot.systems/query, we also now have a query parameter called search that accepts search strings of less than 256 characters. Thank you to @raygervais, @manekenpix, @cindyledev for the review.

https://dev.telescope.cdot.systems/query?search="search string here" should return blogs that contains whatever is entered after the "=" sign. Usually URL encodes spaces with %20 and I thought I would have to decode this, but surprisingly this was not the case and Express.js handles the encoding and decoding(I assume). I also added an error message informing the user if the user doesn't provide / provides an empty string for the search query parameter.

Issue 634: Nginx Configuration for Staging and Production
This one was awesome, I still have no idea what I'm doing with Nginx. But it was extremely fun to just collaborate on this with someone as we were trying to get this to work. Building on top of what @manekenpix had done previously to cache static files within Telescope, we're also caching all endpoints for Telescope. I forgot the exact settings we had, but hitting an endpoint on Telescope will now cause Nginx to cache the endpoint for the next while and instead of having to go to Telescope to get the requested page, Nginx will serve the cached endpoint until it is considered stale.

Assuming there are no cached endpoints yet. We can test it by doing a curl -I https://dev.telescope.cdot.systems/posts, there should be a response header with 'X-Proxy-Cache: MISS'. Visit the address in the browser, then use the same curl command again and this time you should receive 'X-Proxy-Cache: HIT'

Issue 648: Switch from in-memory to Redis-backed Session Management
This was a simple PR, switching away from a package we're using to a production ready package. It was simple until I realized, my PR was breaking a lot of our current tests. A suggestion from @humphd to use our current ioredis library fixed all these issues.

Issue 668 Compare Nginx Config with Mozilla Recommendations
Also Nginx related PR which I have no idea what I'm doing, except to use their recommendations in our nginx configuration file.

Issue 724 Add Site Property to Feeds and Redis
I think this PR is close, I just need confirmation if what I'm doing is a correct way.The feedparser-promised package parses all posts for the processed feed and returns a link in its metadata which is supposed to contain the url without the tags, for example https://c3ho.blogspot.com/feeds/posts/default/-/open-source should become https://c3ho.blogspot.com. But upon further testing, this is not the case and only feeds from Wordpress are working as intended. I wrote a simple function instead to just take the feed url provided by the user and do some regex to obtain the "link" foregoing the metadata link route as it is not consistent.

Issue 750 Make Search Bar Return Results
Building upon work I did previously to create a component for Author results, I now had to combine the GraphQL queries I worked on and the Author component so results will be displayed when a user types a string into the search bar. This was fun and stressful as it taught me more about React hooks, but tons of frustration on trying to get GraphQL queries and Apollo Client to work on the front end. In the end I wasn't able to get search bar to return results when the button is clicked, so I had to opt for it to dynamically return results as the user inputs text. Thank you @cindyledev for religiously reviewing all the later commits for this PR.

Issue 803 Include Version Info on Header Banner
This PR works, we've tested it locally with the commands npm run build and npm run develop and it has worked on several machines. I just don't know why ZEIT doesn't like it. We turned the version info on the banner into a link so when you hover over it, there's the SHA info regarding the commit it is on and clicking on it will bring the user to the commit in Github.

I keep saying this, but for this upcoming release, I'll be finishing up unfinished PRs and get Kubernetes working so I can tackle replacing our REST APIs with serverless functions.

Friday, March 13, 2020

Serverless Functions(Node)

Serverless functions are pretty cool, they take care of another worry developers might have: What if I get so many requests that it overloads my server? Simple, you don't. You let an almost trillion dollar company(Amazon) handle it. These functions are able to scale up and down depending on the # of requests all handled by AWS Lambda.

We'll be using the serverless package which makes settings up AWS lambda pretty simple.

Before we begin, make sure you have a AWS account and user created and provide the user programmatic access:
  1. Use the command: serverless config credentials --provider aws --key userKey --secret userSecret . Replace userKey and userSecret with the appropriate user information you created for the AWS account. 
  2. Use the command serverless create --template aws-nodejs --path folderName . Replace folderName with your choice, this will create a folder with serverless.yml and a file called handler.js
There's really two parts that make up the serverless functions, the serverless.yml and the corresponding .js file containing the functions. For this example I'll have a file called handler.js containing all my functions I wish to make serverless.

When trying to hook up the functions, we must have a few things:
  1. Define the functions in the file(handler.js)
  2. Make sure the available handler is available for the function
In my handler.js file I'll have the two following functions

hello() which returns the message 'Hi' and bye() which returns the message 'Bye'.
Here's how bye looks like

module.exports.bye = async (event, context, callback)  => {
  const str = `Bye`;
  return str;
}

The event argument contains information about other AWS services the function has gone through, if it went through a load balancer it will contain information about the load balancer. For more information about it, find it here.

The context argument contains information about the invocation, function and environment. For more information about it, find it here.

The callback argument contains information you want to send back in case of success or error, the callback actually accepts two arguments callback(response_error, response_success). Amazon provides documentation on how you should handle async vs sync callbacks here

Make sure both functions are exported. In the .yml file under the functions: section you want to create a section for each.
functions:
  hello:
    handler: handler.hello
    events:
        - http:
             path: users/hello
             method: get

  bye:
    handler: handler.bye
    events:
        - http:
             path: users/bye
             method: get
You'll notice we have the function name, followed by handler: fileName.functionName.

To push the code to AWS, we'll have to use the command serverless deploy -v, you'll have to push the code anytime you want the changes reflected on AWS

To call any of the functions to test on command line, use the command serverless invoke -f functionName

To test your app locally, we'll be using serverless-offline package. Once the package has been installed, we have to add the following at the end of serverless.yml:
plugins:
  - serverless-offline

Use the command serverless offline start to start it up locally. This uses default port 3000 and you should now be able to get 'Bye' in the terminal or console when you hit the route: localhost:3000/users/bye

This wasn't too bad. Now you can say you have knowledge of cloud based programming!

Source: https://hackernoon.com/a-crash-course-on-serverless-with-node-js-632b37d58b44

Planning for March Break

Most public institutions have closed or are preparing to close for the next week and now we're getting an unexpected March Break so there's some planning on what to do for the next few weeks while this happens:

Running:
Last October I ran the ScotiaBank half marathon(21 km) a goal of mine since I was young. I've been thinking if this year's half marathon goes well again, I'll try for a full marathon(42 km) and make qualifying in the Boston marathon an eventual goal. I thought I did pretty well finishing in a time of 1:45 for a half marathon until I realized to qualify for the Boston event you need to finish a marathon in ~3:03. This means I have to shave off 15 mins off my half marathon time WHILE running twice as long. I've got a long way to go. The improving weather and off time should allow me to start running earlier.

Boxing:
I've been boxing for ~6 years now and instructor for 2 years. This week or more off should probably allow me to take some of the classes instead of only teaching.

Teaching:
A friend of mine reached out to me around December to see if I would be interested in teaching programming to children(Scratch) and pre-teens(basic Javascript) once a week for 8 weeks starting in at a community center starting February. I'm assuming it went well as he asked me to teach HTML/CSS basics starting April. Time to plan how and what to teach kids about HTML.

On a side note, for anyone who wants to test quick things for Javascript like a quickly written function or to test some packages without wanting to install express package and all that stuff in Visual Studio Code, I highly suggest Glitch. They have an option for launching a basic web-page or a node-express app (allows users to also install packages), they also have a terminal to interact with your app and the option to import/export your code from/to Github.

Dogs:
I have two dogs, got to take them out for more walks or runs during the time off!

Programming:
During the first class of OSD700 our professor mentioned something pretty interesting that we could look into for Telescope, serverless functions. I'm not entirely sure if we'll have the opportunity to use it in Telescope, but I figured I'll look into it and if the opportunity arises, collaborate with another student who has expressed interest in it on and implement it.

Another is to learn a bit more about deployment: Docker and Nginx. I just kind of use Docker to get ElasticSearch running, but have no idea how to create a docker file from scratch myself as all the hard work was done by @manekenpix and @raygervais

Last goal I have is to also trim down the amount of issues I have assigned to myself. I think I had somewhere around 15 issues...

Contains Duplicate (Leetcode)

I wrote a post  roughly 2/3 years ago regarding data structures and algorithms. I thought I'd follow up with some questions I'd come...