While bootstrapping a side project, I wanted to get things out to the public as quickly as possible. Being a backend developer myself with relatively bad CSS skills, I came across PyWebIO (an alternative to Steamlit and anvil) on Hackernews.

PyWebIO is a Python library that provides a series of imperative functions to obtain user input and output on the browser. The input function of PyWebIO is blocking, and the output function will output content to the user in real-time. (documentation)

By default, PyWebIO uses WebSocket protocol for server-browser communication. …

Running Scrapy spiders in your local machine can be very convenient during the early stage of development, but we would eventually need to deploy and run our spiders in the cloud periodically.

In search of free options for running periodic crawling jobs, I have come across a few options available in the market such as Zyte Scrapy Cloud where you can easily deploy, run and scale your web crawlers. However, if you intend to multiple spiders periodically, you would have to start paying.

Goal

By the end of this article, you should be able to deploy your Scrapy project and schedule…

While there are a handful of Python Scrapy tutorials out there, I find that most of the tutorials do not cover these low-hanging fruits that can greatly improve the developer experience of web scraping.

In this post, I will be sharing with you some quick and easy tips that you can use when working with Scrapy which will greatly benefit you as a developer while some tricks here can even help to minimize the load to the websites that you want to scrape from.

TL;DR:

• Use HTTPCache during development
• Always use AutoThrottle
• Consume sites’ API whenever available
• Use bulk insert for…

Today, with the help of numerous open-sourced tools, libraries, frameworks, and many more web scraping solutions, web scraping can seem easier than ever.

While that is true, it can be foolish for someone to not expect any challenges while trying to scrape data from the web. For those who are wondering —

“Why scrape the web”?

Gathering data is incredibly crucial in today’s world. Through web scraping, businesses are able to gain competitive advantages such as:

• Lead generation
• Pricing advantage
• Competitor monitoring
• Market research

If you are looking to start your web scraping journey with Scrapy, I would highly recommend…

Selling “covered calls” has been one of my go-to strategies when it comes to options trading as one can collect premium on top of appreciating stock price that you have already own.

In this article, I am going to explain to you what is a “covered call” (CC) and the reasons why it’s one of my favorite strategies to generate extra income on top of the shares that I have already owned.

Before we begin, this article requires you to understand the basics of options. …

Here’s a quick summary — I started writing on Medium in August 2019 and really started committing to writing an article every week since March 2020. Though I am a software engineer by profession, I focus on writing on these 3 topics:

• Software engineering
• Investing
• Personal finance

While I’m not particularly sure about what’s a good number on Medium is, but a 10,000 monthly view without posting on any social media platforms makes me pretty happy.

So, back to the question, how much did I earn from a year of writing?

Here’s the answer — $67.46 in my first year…

As engineering teams are adopting CI/CD practices, software products are getting delivered faster than ever. Error monitoring comes in handy as it provides developers the confidence to deploy faster while knowing there’s a safety net watching their deployments.

On top of that, having error monitoring helps developers to fix and debug errors faster, in order words, developers can spend more quality time building new high-quality features.

The goal of this post is to document and share some of the issues and tips that I have learned while I was working on setting up Sentry monitoring with Django Graphene projects.

By…

When it comes to options trading, the implied volatility (IV) is just as important as the direction the underlying security (stocks) moves.

In this article, I am going to explain briefly what is IV and its importance when it comes to options. This would also serve as a brief introduction to the options Greeks such as Theta, Gamma, Vega, and Delta.

Before we begin, you should at least be familiar with the basics of options such as call options, put options, DTE, ITM, OTM and etc.

If you are interested in learning about the basics of options, feel free to…