Python love Rust

The rise of Rust in the Python ecosystem

These days it’s amazing how many tools rely on Rust under the hood, but for a Python programmer this goes one step further.

(If you don’t know what Rust is, it’s a modern programming language designed to be as fast as C++ but much safer, since it prevents memory errors at compile time instead of letting you suffer through them at runtime.)

Python: simplicity and ecosystem

Python is an amazing language; it’s simple (as everyone knows) and its error reporting is precise, unlike many other languages. Over time this has built a huge community that has created an ecosystem of products and tools of every kind, something hard to find in other programming languages. If you want to build a proof of concept, there are already tools that make the job easier. For example, if you want to get the translations of a YouTube video, you just install pytube and you’ve got a tool that does it for you. There are countless use cases like this, and not just for trivial stuff, but for very serious projects like image processing or data-intensive libraries.

But Python has a problem in many cases: it’s designed to optimize development, not raw performance. The difference is huge. Do you always need that performance? No; in most cases you don’t, but Python already offers a solution for when you do.

There are lots of libraries that take advantage of high-performance languages like C++ and are used as Python libraries, so Python can do what it does best: write complex business logic and delegate performance to the low level. This is great and has been the standard for a long time. But nowadays, C++ is a very useful low-level language that comes with a problem that tends to slow development down: memory management. It’s the kind of problem those of us focused on high-level programming don’t usually see, but at the low level it requires meticulous handling.

In a high-level language like Python, this is done automatically by the Garbage Collector. The cost is overhead on every operation; every time you access or declare a variable, the computational cost is higher than in a language like C++. Day to day this doesn’t seem important, but if you have an application handling 50,000 requests per second, it can mean a huge infrastructure cost. (If you keep reading, I’ll show you why choosing an efficient tool even on small projects is a good idea for your entrepreneur’s wallet.)

Rust solves this in a very strict but ideal way: it decides not to trust the programmer with memory management and forces you to write efficient code up front. That means the language doesn’t need a Garbage Collector, letting it keep high-level language features with C++-level performance. Engineers have seen a lot of value in this, because beyond performance issues, memory management is a critical security problem; Microsoft says over 80% of security issues come from it. With Rust, a programmer knows their program won’t have these bugs, because it simply won’t compile if the code isn’t safe.

This has had a big impact on the Python world, replacing C++ for building native libraries using PyO3. It has triggered the re-engineering of many products we use every day:

• Pydantic was rewritten in v2 to use Rust.
• Polars, written in Rust, is the natural successor to Pandas.
• Ruff (which I love), Black’s successor, written with linter and auto-fix superpowers.
• Orjson, a JSON serializer much faster than the built-in one.
• UV, the replacement for pip and all its competitors.

And the list keeps growing...

Why Rust matters for Python developers

This got me thinking that for an engineer who uses Python, learning Rust is a big deal. Instead of migrating to a new language like Go, we can write Python extensions in Rust to solve complex problems, picking up modern low-level skills along the way. In fact, this is what got me started on my own journey learning Rust as my new complementary language. These days, picking a new language for my stack is something I do very carefully; there’s so much out there that learning it all becomes unsustainable, and having a strategic focus is necessary.

Plus, I think learning a language like Rust makes you a much better programmer. To be efficient in Rust you have to learn to write it well up front so the code compiles on the first try; it forces you to understand what’s really happening with your data.

The strategic value for companies

If you run a company where Python is the main focus, you don’t need to throw everything overboard and switch languages. I think it’s really worth having one or two developers on the team who know Rust and use PyO3 to write optimized modules that solve specific problems in your systems.

Logic that changes often, doesn’t run constantly, or where performance isn’t critical can stay in good old Python. But it’s worth optimizing with Rust the parts where performance is key, where infrastructure costs climb, and where every second counts for the user experience or data processing. It’s a smart way to scale without losing the development agility Python gives you.

Optimizing from day one: the entrepreneur’s approach

Like I said before, it’s important to choose tools with adequate performance even at the start of a project. As an entrepreneur and engineer I keep hearing the phrase "Don't optimize prematurely", which is fine (it’s one of the things that separates juniors from seniors), but let me tell you that most of the time people "hide behind" that phrase. As an entrepreneur with no investors, paying for servers out of your own pocket, having something efficient from the start does count.

Many engineers who don’t build their own products don’t see this. I’ve worked at big companies with very good engineers who don’t get the importance of managing costs because they work with money that feels infinite. But when you experience it on your own, things change. To illustrate, here’s a real case with watered-down numbers:

A real case: Python vs Go in infrastructure costs

I have a Python infrastructure on AWS that costs 100 USD a month. All good, except the service only makes 10 USD in profit. I wondered what would have happened if I’d used a more efficient language to build that app. I went from question to experiment: I spun up a container with minimal resources that cost me 20 USD a month and ran a load test. With that setup, Python maxed out at 100 users. Then I replicated the service’s heaviest APIs and rewrote them in Go (AI helped quite a bit here). The surprise was that, for the same load of 100 users, the system only used 10% of that 20 USD machine’s resources. In other words, with those same 20 dollars I could handle up to 1,000 users in Go, while to handle those same 1,000 users on the original Python infrastructure, I would have had to scale up to paying 200 USD!

Seeing it that way blew my mind. We usually talk about performance just for the sake of it, but we rarely talk about performance in terms of the money it saves us. Choosing a more efficient language means more monthly profit and more money in my pocket. While I might not pick Go for this particular case, I could have used a framework like FastAPI or Rust extensions to get that performance without giving up developer experience.

Conclusion: profitability and technology

Today I’m seriously considering switching my tech stack purely for the sake of personal profitability. It’s essential to think about which technologies we use; sometimes, without much effort, we can improve performance and make a lot more money along the way.