Read About Tips to Deploy and Schedule Run Your Scrapy Crawler on Cloud for Free In 6 Minutes
Running Scrapy spiders in your local machine can be very convenient during the early stage of development, but we would eventually need to deploy and run our spiders in the cloud periodically.
In search of free options for running periodic crawling jobs, I have come across a few options available in the market such as Zyte Scrapy Cloud where you can easily deploy, run and scale your web crawlers. However, if you intend to multiple spiders periodically, you would have to start paying.
Goal
By the end of this article, you should be able to deploy your Scrapy project and schedule…
Read About 5 Easy Tips to Improve Your Web Scraping Experience Using Scrapy in 5 Minutes
While there are a handful of Python Scrapy tutorials out there, I find that most of the tutorials do not cover these low-hanging fruits that can greatly improve the developer experience of web scraping.
In this post, I will be sharing with you some quick and easy tips that you can use when working with Scrapy which will greatly benefit you as a developer while some tricks here can even help to minimize the load to the websites that you want to scrape from.
TL;DR:
- Use HTTPCache during development
- Always use AutoThrottle
- Consume sites’ API whenever available
- Use bulk insert for…
Read About Lessons Learned From Scraping Over Hundreds of Sites With Python Scrapy
Today, with the help of numerous open-sourced tools, libraries, frameworks, and many more web scraping solutions, web scraping can seem easier than ever.
While that is true, it can be foolish for someone to not expect any challenges while trying to scrape data from the web. For those who are wondering —
“Why scrape the web”?
Gathering data is incredibly crucial in today’s world. Through web scraping, businesses are able to gain competitive advantages such as:
- Lead generation
- Pricing advantage
- Competitor monitoring
- Market research
If you are looking to start your web scraping journey with Scrapy, I would highly recommend…
An Easy Way to Generate Income From Covered Call and PMCC on Your Existing Stocks and LEAPS Options
Selling “covered calls” has been one of my go-to strategies when it comes to options trading as one can collect premium on top of appreciating stock price that you have already own.
In this article, I am going to explain to you what is a “covered call” (CC) and the reasons why it’s one of my favorite strategies to generate extra income on top of the shares that I have already owned.
Before we begin, this article requires you to understand the basics of options. …
10,000 Monthly Views, 3,000 Monthly Reads, 50 Followers Later, Here’s My Thoughts
Here’s a quick summary — I started writing on Medium in August 2019 and really started committing to writing an article every week since March 2020. Though I am a software engineer by profession, I focus on writing on these 3 topics:
- Software engineering
- Investing
- Personal finance
While I’m not particularly sure about what’s a good number on Medium is, but a 10,000 monthly view without posting on any social media platforms makes me pretty happy.
So, back to the question, how much did I earn from a year of writing?
Here’s the answer — $67.46 in my first year…
Read About How To Set Up Sentry With Your Python GraphQL API In 5 Minutes
As engineering teams are adopting CI/CD practices, software products are getting delivered faster than ever. Error monitoring comes in handy as it provides developers the confidence to deploy faster while knowing there’s a safety net watching their deployments.
On top of that, having error monitoring helps developers to fix and debug errors faster, in order words, developers can spend more quality time building new high-quality features.
The goal of this post is to document and share some of the issues and tips that I have learned while I was working on setting up Sentry monitoring with Django Graphene projects.
By…
Explaining Implied Volatility (IV) and The Greeks In Layman Terms In 7 Minutes
When it comes to options trading, the implied volatility (IV) is just as important as the direction the underlying security (stocks) moves.
In this article, I am going to explain briefly what is IV and its importance when it comes to options. This would also serve as a brief introduction to the options Greeks such as Theta, Gamma, Vega, and Delta.
Before we begin, you should at least be familiar with the basics of options such as call options, put options, DTE, ITM, OTM and etc.
If you are interested in learning about the basics of options, feel free to…
Read About the Basics of Options Trading With Examples in Just 6 Minutes
I remember vividly how overwhelmed I was when I was first introduced to options. All I knew was there were call options and put options. Never knew one could sell a call option or sell a put option.
Things got even more confusing when I was introduced with more daunting vocabularies such as — Delta, Theta, In the Money, Out of the Money, Intrinsic Values, etc.
Like many, I was intimidated by the technical jargon that people use. Forget about the technical complexities, I didn’t even know what to look (Google) for. I was helpless.
The main motivation for me…
Apps You Should Use to Earn and Maximize Cashback or Rewards In Singapore and Malaysia
Like many others, I often look for quick and easy ways to avoid paying the full cost for a meal.
At worst, whenever I am paying for the full cost of a meal, I would at least want to get some form of cashback or reward points out of it.
Here, I hope to share with you some of the tricks I personally use for myself in order to gain the most value out of your daily transactions in Singapore or Malaysia, without much hassle in our busy lives.
Disclaimer
Bear in mind that some of the links in this post…
Read About Comparison Between Buying LEAPS Options and Shares in Just Under 7 Minutes
Typically, when you are bullish on a stock, you would purchase shares of the underlying stock and hold it for the long term.
In this article, I am going to talk about how one could magnify their returns using Long-Term Equity Anticipation Securities (LEAPS) while understanding the potential risks involved.
Generally, options traders would put on positions with 30 to 60 days to expiration (DTE), which means the stock price outlook or prediction usually falls within one or two months' time frame.
LEAPS options on the other hand would mean options with an expiration of at least one year or…
