As the lead developer, I emphatically disagree with this. :)
If you'd like to state why you think it's insecure, I'd love to hear it, but security is incredibly important to us and something we've put lots of effort into. I don't deny that there may be bugs (it's an alpha), but by design Sandstorm is a highly secure way to run other people's code.
> and inefficient
While it's true that running lots of small per-user (or per-document) instances of apps is necessarily less efficient than running one large multi-tenant server, it's not nearly as bad as it sounds. Instances of the same app share their code an assets (read-only) and are aggressively shut down when not in use, which makes up the vast majority of the inefficiency. Meanwhile, infrastructure continues to get cheaper...
> If you'd like to state why you think it's insecure, I'd love to hear it
The long-term security of Linux containers has not been well explored yet. There have been exploits against the Kernel found, and there are likely to be more.
Plus, how much effort has gone into hardening the Cap'n'Protocol bridge? Do you have a security expert reviewing the code and looking for vulnerabilities? If so, great! I take that part back wholeheartedly.
I appreciate that you've worked past the Docker failing of not signing and validating files; this is a huge step in the right direction.
> Instances of the same app share their code an assets (read-only)
But not their in-memory caches. Code and shared make up a fraction of their presence in memory.
> infrastructure continues to get cheaper
This has always sounded like a lazy cop-out to me. Yes, infrastructure is getting cheaper, but our applications are getting bloated at the same rate. And if we're running potentially dozens of PostgreSQL instances on a single machine, your infrastructure costs to make all of the apps performant are not going to be cheap.
> The long-term security of Linux containers has not been well explored yet. There have been exploits against the Kernel found, and there are likely to be more.
> Plus, how much effort has gone into hardening the Cap'n'Protocol bridge? Do you have a security expert reviewing the code and looking for vulnerabilities?
Among our advisors are Mark Miller and Jas Nagra, both members of the Google security team (though advising us in their free time, not on behalf of Google). Cap'n Proto is based heavily on Mark Miller's previous work in capability-based security.
Also among our advisors is Andy Lutomirski, a kernel developer who specializes in security and sandboxing. He has been cranking out CVEs against the kernel lately. Most of them haven't affected Sandstorm, due largely to our seccomp filter which Andy himself wrote and continues to work on (see link above).
My own background is diverse but includes a few years working on security at Google.
That said, we have not yet commissioned a thorough security review of Cap'n Proto's own implementation. That is something we plan to do before any 1.0 release (of Cap'n Proto or Sandstorm).
> But not their in-memory caches. Code and shared make up a fraction of their presence in memory.
Depends. If the app is written in C++ or Rust, then the code is mmap'd in (with that memory being shared across instances). The runtime memory overhead tends to be very low if the app is single-user.
For apps written in dynamic languages that parse their code at startup, yes, memory usage is a lot larger -- as a rule of thumb, many apps use around 100MB. One idea we have to fixing this is to checkpoint an app at the point when it first tries to read its per-instance data and restore from that checkpoint on future runs. This checkpoint could theoretically be shared between all instances and mmap'd copy-on-write.
That said, we don't feel this trick is immediately needed. For our upcoming managed hosting service, we've run the numbers and are confident that the vast majority of users will not come anywhere near hitting the resource limits we've set even for the "standard" service level, and we aren't losing money if they do.
> And if we're running potentially dozens of PostgreSQL instances on a single machine
For the scale of a Sandstorm app, it makes tons of sense to switch to sqlite, which mostly solves this problem. :)
That's pretty freaking awesome - thanks for taking the time to point all this out. Might I request that you make some of this information more prominent on your site?
> [memory]
You're still talking about program code memory, not the allocated stacks and heaps. The heaps are the important part to me, because they represent db buffer pools, Redis queues, and cached responses - data which will be duplicated if multiple instances of the same command are run.
> For the scale of a Sandstorm app, it makes tons of sense to switch to sqlite, which mostly solves this problem. :)
Which unfortunately references back to my comment about re-writing apps which come in, in an effort to increase performance.
> Might I request that you make some of this information more prominent on your site?
Yes, we should do that. (Tricky, though -- there's so much information we want people to know, but most people will only read two lines. :) )
> Which unfortunately references back to my comment about re-writing apps which come in, in an effort to increase performance.
We've found that a lot of SQL-based apps support sqlite already. For those that don't, adding support may be some work but it's not a rewrite.
For Mongo-based apps, we actually have a patched version of Mongo that reduces the resource usage pretty well. (Basically we just reduced all their hard-coded "pre-allocate at least this much space" constants.) At some point we'll try to do the same for some SQL database...
> For the scale of a Sandstorm app, it makes tons of sense to switch to sqlite, which mostly solves this problem. :)
Case in point: EtherCalc, which usually runs with Redis storage, deliberately uses the fallback "toy" JSON file storage with Sandstorm, which saves 1MB RAM per document instance and makes migration easier.
This works because there's only a few concurrent writers per document at most, instead of the multi-tenant scenario where there's thousands of concurrent writers at any given time.
As the lead developer, I emphatically disagree with this. :)
If you'd like to state why you think it's insecure, I'd love to hear it, but security is incredibly important to us and something we've put lots of effort into. I don't deny that there may be bugs (it's an alpha), but by design Sandstorm is a highly secure way to run other people's code.
> and inefficient
While it's true that running lots of small per-user (or per-document) instances of apps is necessarily less efficient than running one large multi-tenant server, it's not nearly as bad as it sounds. Instances of the same app share their code an assets (read-only) and are aggressively shut down when not in use, which makes up the vast majority of the inefficiency. Meanwhile, infrastructure continues to get cheaper...