The way I understand it: Bytes are just bytes, until you provide an encoding. Then they can be can be converted to a string, if validly encoded. Taking an array of characters and just treating it or casting it as a string is usually a bad idea.
The thing I think Rust maybe goofed, or at least made a little complicated, is their weird distinction between a String and a str (and a &str). As a newbie learning the language, I have no idea which one to use, and usually just pick one, try to compile, then if it fails, pick the other one. I'm sure there was a great reason to have two types for the same thing, that I will understand when I know the language better.
If you want to understand more deeply, the Rust Programming Langauge, chapter 4, uses String and &String and &str to talk about ownership and borrowing. Here’s a link to the start of that chapter: https://doc.rust-lang.org/stable/book/ch04-00-understanding-...
Your blog post is practical and clearly explains what to do, when, which is helpful. What's confusing is why Rust has the two types and why the language designers decided it was a good idea to have to convert back and forth between them depending on whether it was going in a struct or being passed as an argument. I suppose the "why" is probably better found in the Rust docs.
As a long-time C++ user, it seems like std::string vs const char* all over again, and we somehow didn't find a better way.
Yep, that’s exactly it: I wanted to focus purely on what to do, rather than weigh it down with what’s already in the Rust book.
It’s closer to std::string and std::string_view. But yes, in a language with value and reference semantics, when you also care about performance, you just can’t do any better: you need both types. Or at least, if you want the additional correctness guarantees and safety provided by communicating ownership semantics in the type. C gets away with just char * but then you have to read the docs to figure out what you’re allowed to do with it and what your responsibilities are.
A Rust `String` reference (i.e. &String) can always be passed where `&str` is expected because `String` has a `Deref<Target=str>` impl... in that sense they don't just appear similar, they are polymorphic.
There may not be a single encoding for every byte in a string. The encoding may not be knowable ahead of time. You might be trying to extract strings from a random blob of bytes with unknown origin. There's a thousand and one different variations.
To give a real example, I once wrote some python scripts to parse serial messages coming off a bus. They'd read the messages, extract some values with regex, and move on.
Unfortunately the bus had some electrical bugs and would intermittently flip random bits with no CRC to correct them. From my point of view, no big deal. If it's in something outside the fields I care about, I won't notice it. If it's flipped something I do care about we have a bad sample to drop or noise the signal processing will deal with. Either way, it's fine. Python on the other hand cared very much. I rewrote everything in C once I got sufficiently annoyed of dealing with it and more importantly explaining to others how they couldn't "simplify" things using the stdlib APIs.
// NB: Must be utf-8!
struct string {
size_t sz;
size_t capacity;
unsigned char *buffer;
};
&String in Rust is roughly like `const struct string *`.
str in Rust is just an array of (guaranteed utf-8) unsigned bytes. It does not have a capacity, so it can't be resized. You can't directly construct one (on the stack), because its size is undetermined and Rust doesn't have dynamic-sized stack allocation.
&str, and Box<str>, are pointers to str, along with a size, and are roughly like this C:
// NOTE: Must be utf-8!
struct str_ptr
{
size_t sz;
unsigned char *buffer;
}
The difference between &str and Box<str> is that the latter is an owned pointer to a heap allocation which will be freed when it goes out of scope. &str is unowned and might point anywhere: to a Box<str> on the heap, to a String on the heap, or to read-only static memory.
IMO, it's probably easier to first try to understand the difference between `Vec<u8>`, `&[u8]`, and `&Vec<u8>`, because they are slightly less "weird" than the string types: they aren't syntactically special like `str` is[1], and they don't have an implicit requirement to be utf8 that is inexpressible in the type system.
[1]: `str` is syntactically special because it is basically a slice, but isn't written in slice notation.
String/str are both valid UTF-8 by definition, though. Plain ol' piles of bytes in Rust are generally represented by Vec<u8>/[u8].
Rust could have done better in naming, but a definite design goal of the language (for better and worse) is to not make things that are complicated for the compiler appear simple to the user. Which unfortunately results in:
The thing I think Rust maybe goofed, or at least made a little complicated, is their weird distinction between a String and a str (and a &str). As a newbie learning the language, I have no idea which one to use, and usually just pick one, try to compile, then if it fails, pick the other one. I'm sure there was a great reason to have two types for the same thing, that I will understand when I know the language better.