I have difficulty accepting "let's replace C with X", where X is a memory-managed language. As a systems programmer (I write SCSI driver code in C), I can't overemphasize how important it is to be able to address memory as a flat range of bytes, regardless of how that memory was originally handed to me. I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs. If your memory manager would not allow this or would move this memory around, that's a non-starter.
I don't stick with C because I love it. If I'm writing something for my own purposes, I use Ruby. I've written some server code in Golang (non-production), and it's pretty nifty, even if the way it twists normal C syntax breaks my brain. I even dabble in the dark side (C++) personally and professionally from time to time. And in a previous life, I was reasonably proficient in C# (that's the CLR 2.0 timeframe; I'm completely useless at it in these crazy days of LINQ and the really nifty CLR 4 features...and there's probably even more stuff I haven't even become aware of).
But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. And even if they did, the pain of rewriting our entire operating system in Go or Rust or whatever would be way more than the alleviated pain of using a "nicer" language.
(We never use 'int', by the way. We use the C99 well-defined types in stdint.h. Could this value go greater than a uint32_t can represent? Make it a uint64_t. Does it need to be signed? No? Make sure it's unsigned. A lot of what he's complaining about is sloppy code. I don't care if your compiler isn't efficient when compiling sloppy code.)
Having written a lot of driver/kernel code and also dabbled in Rust, I have a hard time seeing anything that Rust can't do in this regard. All of the memory manipulation you will typically do is doable in Rust, though you will certainly lose some of the safety if you're bouncing things between types. The only real limitation you'll have is the inability (AFAIK) to do inline assembly.
There'd be no need to rewrite anything to work with Rust; the binaries emitted by the compiler should be just fine, assuming ABI compatibility. Maybe some changes to the way things are linked?
This is what's been so off-putting about Rust, to be honest.
"I should never assume" is precisely what bothered me.
I loved Rust at first, but it required so much maintenance with its constant changes between versions, that I stated to dislike it. I didn't have much time (and I enjoy learning new languages), but when I had to virtually unlearn something just learned the previous day, it became frustrating after a few iterations.
A colleague at work once sneered at me when I suggested I could do my next [internal] project in Rust; I didn't end up using Rust, because I got self-conscious about the idea, and kinda got scared that I'll have to do much, much more work simply because completely valid code would not work in a few days. That feeling is kinda stuck with me to this very day, even though Rust wasn't a stable release back then, and now is.
Not to mention, there's people I've talked to about Rust who still feel like Rust isn't mature and never will be—simply because of it's "reputation."
[1]: They were referring to how dramatically and chaotically it changed before the first stable release.
It's kinda bad, but the other options are worse. What're ya gonna do?
(Now, could we do something about the fair-sized chunks of the standard library that are marked "unstable"? I'd be happy with a compiler flag to turn that error into a warning....)
I wonder how many of the features in Rust that have changed were designed with that possibility in mind? And if not, whether it would have made any difference if they had been?
As a very drastic example, the runtime was only actually removed six months ago. If we still had it, our 'embedding Rust in other languages' story would be significantly worse, and the kinds of things being discussed in this thread wouldn't have been possible.
The goals of Rust have not really changed since its development.
The fact is that (in particular) a region and lifetime system are hard. We were doing research. Research usually requires many attempts to solve a problem.
It feels like languages often struggle with compatibility issues down the road because they assume that they're making all the right decisions from day one. Since you guys are aware that getting this stuff right takes many attempts maybe it would factor in the design process of a new Rust feature that it might eventually be replaced? I can understand if it isn't though, it's another constraint on problems that are already very hard.
Many times during Rust's design process, I saw quotes like this:
"Rust is not the first systems language and will not be the last."
and
"Rust is a language designed for the hardware of today, not that of ten years from now."
Rust is just a stepping stone on the path to better systems languages. It's not the greatest language ever and it made mistakes, but it gets enough right to make it a very compelling option in the systems programming space.
Why would you expect stability from software in early alphas is beyond me. If you want to be an early adopter you need to take the maintenance problems into account. And if you just wish to use a somewhat completed language you should have just waited for a final release.
PLs are complicated beasts and developing them takes time. It's normal for a language to change drastically during development: frequently it turns out that some feature doesn't fit well with some other, and you need to scratch the feature or redesign both features.
For what it's worth, inline assembly has been available in Rust ~forever, it's just not in Rust 1.0, the stable release, so a lot of people missed the existence features like this.
On the flip side, Rust 1.0 doesn't have any of the problems you describe. Rust 1.0 code will work just fine in Rust 1.300 when it comes out in 2050, provided that the language is still around. And inline assembly will end up in a stable release as soon as that promise can be made about it.
Inline assembly landed in Rust 0.6, April 2013, so if you haven't checked it out since then, there's a _lot_ of new/different stuff :)
We've been discussing stabilizing these kinds of low-level features: no_std is first up. We're not 100% sure inline asm is worth it, as you can always link directly to something that's just written in assembly, but we'll see.
I guess my mental estimate of last having looked "about 6 months ago" was off by a little bit. I think this weekend might be dedicated to writing a Rust kernel. Thanks for your hard work!
No problem. Like I said, we're very interested in making this use-case better, so please file any odd things that crop up. Oh, and there's #rust-osdev on mozilla's IRC where friendly people idle.
I suspect that the overwhelming majority of cases that warrant inline assembly would not benefit if it was instead a function call out to something you've linked in. That is, most inline assembly is a small snippet of optimized code that necessarily exists in the function you're optimizing.
Yeah, as always, when we're considering stabilizing or removing something, we talk to who is using the feature and in what way, and these kinds of issues surface.
Inline from C libraries won't -- as far as I'm aware -- inline into a Rust function. Inline is a compile-time hint, not a link-time behavior. In theory this could be changed in LLVM, but it seems really unlikely to me. The function exposed from C (with the inline asm) can be naked, though, assuming you are hygienic.
Hmm... Because 32-bit Windows IRQ handler doesn't save FPU/SIMD state, Rust also needs to have ability to suppress any FPU/SIMD output, when generating code to run under 32-bit Windows kernel at DIRQL (= interrupt handler). Otherwise the mayhem in userland will be rather interesting, usermode thread SIMD state gets corrupted.
It'd definitely be tricky to use Rust for kernel drivers, but still so tempting!
Hm, I bet there's a flag to disable SIMD entirely (not really important here), and maybe even enable softfp if you use any floats in your code (or libraries, by extension). That'd be interesting to hack in, if it's not already there.
I've written C almost every day at work for ~7 years, and I have maybe written the keyword 'float' half a dozen times.
I do embedded and systems development - you never need floating point in that sort of environment. It's only when you get into application domains that you need it.
There are a number of architectures where you leave half of the performance of little things like memset and memcpy on the table without SIMD.
as a footnote: everytime in my career that someone has said "you never need floating point in X [kernel/network stack/some embedded thing/boot/whatever]," they've come back within a year to ask for my help with floating point library routines for X. So now you've cursed yourself =)
By intrusive data structures, I mean data structures where the per node metadata is stored within the struct of the data rather than externally. Linux's list_head is a good example of this. Having one less pointer indirection can make it a bit more cache friendly.
And yes, Rust can handle non tree graphs as well as any other unmanaged languages, but the heavy use of 'unsafe' makes me feel icky. : )
I switched over to using size_t for array indexes a few years ago, and found this a big improvement. And once I switched to size_t, most of my uses of unsigned went away - either you care about the size (and you want a uintN_t), or you don't (and you probably want size_t).
I think this works best if you pass a pointer to an error structure in the parameters. I've found people often use signed integer types with arrays so they can return a -1 to signal that something went wrong.
> But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend.
Oberon, Modula-3 and D allow for it via their SYSTEM/Unsafe/@system modules, but the two former ones failed to get a dent into the OS market (for various reasons) and D still has some improvements to their memory model going on.
Also Ada and SPARK are usually the languages to reach for in life critical systems.
Also lets not forget before C became widespread outside UNIX, Modula-2 and Pascal dialects were saner alternatives.
Are you on a 64-bit target? Then a uint32_t gets converted to a 64-bit signed value before any arithmetic operator is applied. There are some real pitfalls here, though they are often exaggerated.
Do you turn off the TBAA in your compiler? In my experience most systems programmers either turn it off, or don't know the rules.
[edit] I forgot int is typically 32-bits on 64-bit targets. The same argument would still apply for uint16_t and smaller though.
The intersection of people who are willing to rewrite all their arithmetic to use such a library with people who are not willing to switch to a non-C language is rather small.
Most projects still written in C are those that make extensive use of C libraries. Making the application code immune doesn't actually reduce the vulnerability surface much - much of the vulnerability comes in the libraries the application calls.
A lot of people seem to assume that Chris (the author) was talking about managed memory, which he never mentioned once. Managed memory is runtime safety, a type system is compile-time safety. He's complaining about the type system. As an example:
> address memory as a flat range of bytes [...] I can then typecast into byte-aligned structs
You should never have to do that. You shouldn't be able to do that. Your job should be far simpler. Look at unique_ptr: a whole class of bugs are eliminated by this ZERO cost abstraction. Possibly what Chris is advocating is being able to describe what an I/O port is to the compiler and then using that abstraction to write your SCSI driver. This intent should be compiled down to as-good machine code (if not better) than what your C compiler would have given you - in the same way that unique_ptr is compiled.
I don't think any existing language gets this right.
> You should never have to do that. You shouldn't be able to do that.
Someone, at somepoint, has to do this though. Custom memory allocators are more or less predicated on having a byte buffer you chop up and use like this.
I think the best we get -- especially in driver code -- is well thought out design that have low cost abstractions between the device details and the application logic. But that seems like a library detail more than a compiler or language one.
Agreed. I was pointing out that my interpretation of the email was a hypothetical tone, instead of a factual tone. As far as I'm concerned C isn't good enough, but I generally keep my mouth shut about it because I can't offer anything constructive to the discussion: I don't know what a "good C" would look like.
Or even worse -- for example, ARM CPUs usually round-down the misaligned address to the closest boundary when alignment checking is disabled. This means that attempting to access a 4-byte int at location 11 will silently let the CPU access it at location 8. This can manifest in some very nasty bugs.
Yes, it is a very common crap indeed. I had a lot of pain porting some Linux filesystems to Sparc, for example (had to give up back then, that code was beyond any hope).
Not to mention the endianess issues, an often sight in the crappy networking code.
I agree, and I'd like to add that its not just this particular author, but most people who criticise C about it's "insecurities" use sloppy code when they criticise C, which always bothers me. I'm far from being a C fan (I'm also a Ruby fanboy), but programming languages aren't safe, only code can be safe, and that depends entirely on the developer.
Yes, it's "easier" to introduce some bugs in C than Ruby (or Go, or whatever), but that's because whoever wrote that code with the bug didn't know C well enough. Is that C's fault? Same can be said about any language, really.
If you don't know that String#match returns nil on unsuccessful matches and try to call MatchData#[], you'll get a NPE (something along the lines of "undefined method `[]' for NilClass"). This is very similar to dereferencing a NULL pointer in C[1].
[1]: I know dereferencing a NULL pointer in C is undefined behaviour, but your program will crash—if you're lucky enough—when you try to work with NULL pointers when you don't expect them.
This is nonsense. C has a very weak type system and very weak runtime guarantees, making it much easier to introduce problems with no indication that something's up. Other languages with strong type systems and/or stronger runtime checks eliminate large classes of bugs that are very easy to trigger in C.
So, yes, it is C's "fault" that it doesn't protect against classes of bugs that many other languages do. Sure, those languages have some of the same bugs that C does, but they're missing most of the very worst ones and that's really powerful. For example, a garbage collector protects against accessing dangling pointers: it's just not something the programmer has to worry about at all.
Rejecting cricitisms of C's safety inadequacies with "just code better"/"just learn the language better" doesn't work in practice: there have been too many high-profile vulnerabilities in C software, many of which would've been much harder to trigger in other languages.
float _Complex -> float doesn't require any diagnostic even though the imaginary component is (silently) discarded. Clang has one (not enabled by default or -Wall/-Wextra) however current GCC versions haven't.
I think you're discounting the fact that the software with the bugs was written in C precisely because C was a better fit than the other languages which offered greater protection.
I'm not saying (and have never said) that C is better than other languages for every problem. I'm just saying that I read too many articles about the "problems" with C that Stupid C Programmers must not realize since we keep using it where the author of the article hasn't bothered to understand why programs are written in C in the first place.
Since the author of the post is also an author of LLVM, clang, and Swift, and the director of developer tools at Apple, he certainly understands the difference between sloppy and non-sloppy code, but he also knows that sloppy code is a reality and undefined behavior is dangerous.
So we can make everyone a better programmer, or we can make better languages, or we can throw in the towel and say things are good enough. I think he's suggesting the correct path.
The undefined behavior of C and C++ results in remote code execution to a degree completely unmatched by other languages.
Examples of dangerous UB in C always use deliberately sloppy code for pedagogical reasons. For real-world examples of problems, look at the CVE database.
"Lack of other languages" was poorly worded. I'm sorry.
What I meant was that, after the spread of Unix and C, C is the most common systems language in production. (Before the spread of C, I believe assembly was the most common systems language(s).)
I suspect that any systems language would have a bad reputation if it were used (and over-used) as much as C.
Lots of languages around the time C came out had array bounds checks at runtime. C has a hard time doing that because of pointed and arrays being (mostly) interchangeable.
> Yes, it's "easier" to introduce some bugs in C than Ruby (or Go, or whatever), but that's because whoever wrote that code with the bug didn't know C well enough. Is that C's fault? Same can be said about any language, really.
I thought the promise of computers is that we didn't need to have smart people working on repetitive, boring, error-prone jobs.
And yes, there's a line between languages like C, Ruby, Python, Objective-C, etc. on one hand that don't actively try to make bugs hard, and Ada, Rust, Haskell, Ur, etc. on the other. That line is not particularly lined up with something like interpreted vs. compiled or old vs. new, and if you look for the line there, you won't see it.
I don't know anyone who can write proper, secure, bug free C code in multithreaded environment. Some people, such as DJB [1] do get pretty close, though.
Much less those who can do same with C++.
But when you have larger teams, it gets even harder. People just think so differently and misunderstand intentions without realizing.
I did think I could do that in my twenties. 15+ years later I have a lot more respect for C.
> most people who criticise C about it's "insecurities" use sloppy code when they criticise C, which always bothers me
If you're really really lucky, your coworkers will only write sloppy code by accident. But unless you're only working on toy projects, statistics will catch up to you and sloppy code will happen. To err is human.
By NASA standards, I suspect most of your code has been written "sloppy". As has most of mine.
> but programming languages aren't safe, only code can be safe, and that depends entirely on the developer.
Languages can be safe in the sense that they can force code to be safe in specific ways, or at least warn you better with unsafe opt-ins or better static analysis.
We agree that the developer is to blame for the thousands of overflow CVEs out there.
One developer recognizes they're not an infallible robot, nor are their coworkers, nor is the new intern they're about to hire, and uses the tools at their disposal - static analysis, "safe" languages, etc. - to catch and fix some large percentage of certain mistakes they, and those they work with, make.
Another developer scoffs at the first for "blaming their tools" and tries to avoid mistakes with sheer willpower. By not setting up static analysis, maybe they save enough time to do an additional 10-20 code reviews over the course of the project.
All else being equal, who will end up with safer code?
> Yes, it's "easier" to introduce some bugs in C than Ruby (or Go, or whatever), but that's because whoever wrote that code with the bug didn't know C well enough.
If this is true, nobody knows C well enough. Find me a programmer who's written a sufficiently large C project without a single bug, and I will worship them as a living god.
> Is that C's fault? Same can be said about any language, really.
I don't care about fault, per se. But sure, let's blame C. And every other language. Let's not blind ourselves against their faults, and the possible ways we might improve them, and the possible ways we might adapt ourselves to them.
Let's not saddle ourselves with stone axes for the rest of our lives.
> If you don't know that String#match returns nil on unsuccessful matches and try to call MatchData#[], you'll get a NPE (something along the lines of "undefined method `[]' for NilClass"). This is very similar to dereferencing a NULL pointer in C[1].
Hence the point of talks such as "Null References: The Billion Dollar Mistake", and why some languages are designed to avoid letting you access potentially null/nil/nothing variables without checking that they aren't first.
> I have difficulty accepting "let's replace C with X", where X is a memory-managed language.
What about C++ (which adds RAII, which I believe is indispensable for writing correct code, especially over C) or Rust, which adds much better memory correctness? I'm in agreement where the article says,
> C, and derivatives like C++, is a very dangerous language the write safety/correctness critical software in, and my personal opinion is that it is almost impossible to write security critical software in it
Though I believe it can be done in C++, with some discipline (but much less than C would require).
> As a systems programmer (I write SCSI driver code in C),
I think SCSI driver code counts as a niche application
> I can't overemphasize how important it is to be able to address memory as a flat range of bytes, regardless of how that memory was originally handed to me. I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs.
My understanding is that C doesn't generally allow this; that's what the strict aliasing rule is, and what's "wrong"[1] with several of the examples in the article. IIRC, you can get a [unsigned] char * into a struct (but why?[2]), but attempting to cast a char * to a struct foo * is forbidden.
(Of course, with amends to the thread's original purpose, which is asking what the common layman understands / uses / depends on. Type aliasing is not well understood in my opinion. I'm not entirely confident I've got it right in this post.)
> If your memory manager would not allow this or would move this memory around, that's a non-starter.
(same comments about Rust/C++)
> We never use 'int', by the way. We use the C99 well-defined types in stdint.h. Could this value go greater than a uint32_t can represent? Make it a uint64_t. Does it need to be signed? No? Make sure it's unsigned. A lot of what he's complaining about is sloppy code.
In my experience, this is a rare thing; especially while interviewing, I find the majority of candidates — claiming to be most comfortable in C (we allow language of choice, in the hopes that you choose your strongest!) — don't know what `size_t` is.
[1]: The array copy code is correct, but the author is lamenting optimizations that cannot be taken unless we assume the pointers don't alias; the int-to-float code is UB (hence why he writes "miscompile" in quotes; it's UB, so by definition there's no wrong output (though an error might be nice); this is also why "obvious" is in quotes: humans know what the programmer meant, but what the programmer wrote is UB; I think this is telling about C: human expectation and the language don't align, from a language-design standpoint, this is not good).
[2]: most of the time I see people reaching for a char-pointer-into-a-struct, or cast-char-pointer-to-struct, they're short circuiting actually decoding some I/O byte-stream into an in memory data structure. This is not portable, unless — maybe — if you do "packed" structures (which is still not portable, I believe), but then you're sacrificing performance by potentially having unaligned members in the struct (which are harder for the processor to deal with, and might require multiple (e.g., MIPS) or unaligned (e.g., x86, amd64)) loads/stores.
I don't stick with C because I love it. If I'm writing something for my own purposes, I use Ruby. I've written some server code in Golang (non-production), and it's pretty nifty, even if the way it twists normal C syntax breaks my brain. I even dabble in the dark side (C++) personally and professionally from time to time. And in a previous life, I was reasonably proficient in C# (that's the CLR 2.0 timeframe; I'm completely useless at it in these crazy days of LINQ and the really nifty CLR 4 features...and there's probably even more stuff I haven't even become aware of).
But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. And even if they did, the pain of rewriting our entire operating system in Go or Rust or whatever would be way more than the alleviated pain of using a "nicer" language.
(We never use 'int', by the way. We use the C99 well-defined types in stdint.h. Could this value go greater than a uint32_t can represent? Make it a uint64_t. Does it need to be signed? No? Make sure it's unsigned. A lot of what he's complaining about is sloppy code. I don't care if your compiler isn't efficient when compiling sloppy code.)