This. I'm SUPER excited to have a Swift backend for a web app, and/or I've been toying around with the core logic of another app. Either idea doesn't need anything in the way of GUI, just the standard library and maybe a few things from Foundation that should be in the standard library anyway (like pseudorandom number generation).
I think Foundation might be necessary though, even if you just use Swift structs (Array, Dictionary, etc)