I’d add that most of the time nobody actually measured anything. at work we inherit slow sites all the time and the first guess is always “the stack”, then you profile it and it’s one query running per item in a loop or something like that. I had the same with my own side project, was sure I needed a bigger server and after measuring the problem was somewhere else completely. measuring first is boring but it’s cheaper than any rewrite I guess
Right now everyone is asking their AI agents to rewrite programs in Rust or C or assembly. This is fun, but it’s not a serious attempt at optimization.
C or assembly seems pointless and only for performance. But the reason people (re) write programs in Rust is because Rust as a language has a lot of guardrails that help make a program more stable, more correct. That reliability is the real reason for Rust. The performance is just a cherry on top.
This is actually exactly what you want in a language for AI. This is anecdotal from my side, but in my experience, an AI writing Rust will write far less sloppy crap than an AI writing Python.
Most of the argument against Rust has historically been that prototyping is slow and the language is a bother to write because the compiler makes it hard. But this only holds if humans are the bottleneck for writing the code! Rust therefore has all of the benefits but basically none of the downsides when having an AI write it.
But the reason people (re) write programs in Rust is because Rust as a language has a lot of guardrails that help make a program more stable, more correct. That reliability is the real reason for Rust.
Yeah, but then the Node-like atomic packaging runs contrary to optimization and security by shoving 500 MB of code in your cli-tool. And sure, the compiler optimizes 90% of it away. But that’s not how you build stable software.
I agree that dependencies and package management is a real problem in programming nowadays, but it is not a problem unique to Rust. In fact, I struggle to name a single popular language nowadays that doesn’t have the exact same problems.
It’s not so much “But that’s not how you build stable software.” It’s more like “But that’s not how you build software [at all today].” Modern software is powered by loads of shared packages, that’s just how all modern languages work. This is equally true for Rust and Python.
Rust still has the edge here btw. Rust libraries (crates) are often very high quality with very good documentation. The language again helps here a lot, by providing extensive tooling for automatic documentation generation. Documentation in other languages are is far less helpful or poorly written. This is also again something that helps an AI write better code (just as it helps humans write better code).
The packaging also helps elevate Rust to a high-level language, because you can build easily usable abstractions in packages that can be used in simple programs. All the complexity can be hidden away in a crate and suddenly Rust feels more like Python. I think that’s actually a good thing from package managers.
I wrote a program in Python to read some huge Json files. It took forever to finish. Turned out I could optimize it with some simple changes, which reduced 10 minutes to 10 seconds (not literally, just figurative) and made a difference in huge files only. Later I rewrote the program in Rust. And you know what happened? It was slower than the Python version. Only after understanding what I was doing and making right changes made the Rust version faster.
Most of the time its the design of the code or libraries that slows down. The language might play a huge role in performance, but it depends on what type of code and application it is.
I actually wanted write a blog post about this… if I still have the project data I might explain what happened. BTW this had nothing to do with Ai.
Counterpoint, I wrote a Python program that reads a load of JSON and runs regex searches over it. It was quite slow. Rewrote it in Rust as a straight 1:1 port, no algorithmic changes. 50x speedup.
I think you must have either done something very wrong or your Python program didn’t really do anything other than call Python built-ins that are actually written in C.
Did you use a profiler to find the bottlenecks? Did you try using cjson to speed up the json parsing? I would also avoid using the stdlib regex library too. It’s probably more of a convenience/reference implementation, not something to be intended for high performance. Maybe there’s a re2 wrapper for Python or a similar regex library in C with Python bindings.
Rewriting in Rust is fine, but good Python devs know how to find bottlenecks in their software and replace them with fast implementations. I’ve written plenty of performant Python code, it’s not very hard to profile and optimize. Usually the performant stuff is already written in C/C++ or Fortran anyway (numpy, PyQt, etc.)
Did you use a profiler to find the bottlenecks?
No. It was a simple enough program that there were no areas for improvement anyway. I also did not use a profiler for the Rust code.
good Python devs know how to find bottlenecks in their software and replace them with fast implementations
In my experience this is also a sweet sounding lie to make people feel ok with using Python. Yes in some case you make a big mistake and there’s a large bottleneck you can easily remove… Or your program has a nice small hotspot where it spends most of its time and you can just write that bit in C++ (AI is the obvious example).
But most programs are not like that. They’re more like MS Word. Slowness is evenly distributed through the code. There is no hotspot to remove.
As an example of this, look at Nicholas Nethercote’s excellent series of blogs on making the Rust compiler faster: https://nnethercote.github.io/
The “bottlenecks” they find result in 1% speedups.
If you didn’t use a profiler, then how can you be sure that the issue was Python and not something else? If the slowness really is evenly distributed through the code as you suggest, then a profiler would show that almost instantly and it’s very easy to do that in Python using cProfile.
On top of that, the Python standard library is built for convenience, not performance. The nature of it being a standard library means that the implementation has to be simple. fastregex uses LLVM and SIMD for example, that’s something that’s wildly impractical for the Python stdlib to support. In contrast, I have no idea what regex or JSON library you used for your Rust code, but if it were outside the stdlib, chances are high that it had much more leeway to optimize performance because it didn’t have all the other maintenance obligations that stdlib code needs to support.
All of this is just pure speculation on my part, but ultimately I’m trying to make a point that it is hard to say that the difference was caused by a language issue and not something else. There are lots of other things that could be contributing to this difference and you can’t really say for sure without meaningful diagnostics.
As an example of this, look at Nicholas Nethercote’s excellent series of blogs on making the Rust compiler faster: https://nnethercote.github.io/
The “bottlenecks” they find result in 1% speedups.
The kinds of software that you and I write on a daily basis is nothing compared to pieces of software that get millions of man-hours per year. Assuming you’re not Linus Torvalds or Ted Ts’o hiding behind a funny alias, of course. I don’t think anyone is suggesting that it’s a good idea to write a compiler in Python (outside of prototyping purposes perhaps) since it’s obviously not I/O bound, but if you and I are trying to optimize something we wrote ourselves, we’re more likely to be in the “nice small hotspot” domain you described, rather than the 1% bottleneck speedups discussed in the link you posted.
Very cool link though by the way, I look forward to reading as much of it as I can…
If you didn’t use a profiler, then how can you be sure that the issue was Python and not something else?
It was a relatively simple program and I am good at programming.
The point is, I put the same amount of effort into writing the same program in Python and Rust, and Rust was 50x faster. With AI, I don’t see any reason to use Python these days anyway. I never bought the productivity arguments before (Rust is easily as productive as Python), but with AI Rust is definitely more productive than Python because of the much stronger static checks.
Has everyone forgotten the art of creating or using performance profiling tools
I was in a meeting with a senior software engineer, who makes a shit ton more than me. While sharing my screen he saw flame graph I had for one program I was working on. They asked “what is that?”, I was taken back, and said it’s for looking at which functions are taking more time. Then I was asked “where did you find it?” I was shell shocked, didn’t know how to answer that. Similar things have happened before where I taught them certain things, and they go “great find”, like dude, I didn’t find it just now. That’s basic knowledge if you work with database.
I am not even a developer hire, I’m self learned, and because the company doesn’t have many developers they gave me some of the tasks and I’m doing it because it’s more fun than my job responsibilities, and when I don’t have enough tasks.
You see, performance is rarely a talking point in retrospectives unless its egregious. If you say a piece of compute is struggling the solution is to always buy more compute. It is rare that anyone would think of optimization unless you’re writing libraries.
Or if its employee facing only let it get slower and slower to the point where it takes a minute to update one record
