Hey Pythonista,
Has this ever happened to you?
You find yourself spending a lot of time optimizing a piece of code, only to realize over your next coffee break that you've been optimizing the wrong thing?
You've been so focused on making the code faster that you didn't realize that the real bottleneck was somewhere else.
I fell into this trap yesterday. I retrieved some customer feedback data from the Pybites Coding Platform yielding a 277M JSON file.
I loaded it in, filtered the comments and saved it to a new file. It was a bit slow, which got me down the use-rust-to-make-it-faster rabbit hole, which in itself was a good exercise (I am learning Rust after all), but it was also a bit of a waste of time.
Because when I stepped back, I realized that I was optimizing the wrong thing in the first place!
The bottleneck was not the JSON parsing, but the fact I had to filter the data. I could have saved myself time by focusing on better preparing the data at the source.
I had used Django's `dumpdata` command and it already felt wrong that I could not filter on a single column. Had I researched a bit more I would have found I could have accomplished that using:
Using one of these techniques I would have reduced the JSON from 277M to 7M resulting in a much faster experience.
---
This example is trivial, and the slow performance was still acceptable, but it highlights a good lesson: keep the bigger picture in mind when you're working on code, be it optimizing or any SW problem really.
And the only way to do that is by stepping back and taking a break from the code! It's easy to get into tunnel vision which can potentially cost you a significant amount of time.
---
Have you ever hit this? Reply and let us know, or better, join our community and post about it there:
https://pybites.circle.so/home
Bob & Julian
P.S. In order to hit these kinds of issues and have their subsequent realizations, you actually need to code! Reading books and watching videos is just the start.
We help people achieve their goals by holding them accountable to coding daily on real-world projects.
Are you looking for more intensive training methods with a strong 1:1 component (something bootcamps unfortunately don't offer) then come talk to us about coaching.
Take your career in your own hands, apply here: https://pybit.es
|