Hi Mitchell, it's really great to have you explain all of these things here. Thank you both for this and for writing Vagrant, there is no doubt that it is a valuable piece of software. I have a few more questions if you don't mind:
1. When you first started with Vagrant, did you consider using Python at all? Did you consider using libvirt? Because VirtualBox offers a Python API out of the box (or it does now) and libvirt comes with direct support for Python (http://libvirt.org/bindings.html). I like Ruby and generally I prefer it to Python but I am interested in your view of whether it was the right tool for the job in this case or whether the language choice didn't matter much.
2. In your view what is the overlap in functionality between libvirt and Vagrant and what are the differences? As far as I can see libvirt does let you spin up boxes, talk to them and tear them down. The notable difference is that there is no Puppet of Chef provisioning, is there anything else beyond that?
3. veewee in version 0.3.0 (currently in beta) seems to offer the same functionality as Vagrant plus the ability to create boxes using templates, minus the ability to run provisioners. If they add that say in 0.4.0 then it will be 1:1 in terms of features with Vagrant. Any thoughts on this?
Thank you again for your time and for answering in such detail :-)
No problem, I'm glad you're getting value out of the responses. In response to your questions:
1. When I first started Vagrant, I was a full time Ruby developer, so Ruby was really the obvious choice for me. I don't really see language as a barrier for using a tool, my opinion is generally use the best tool for the job. And as a Ruby developer that at the time had 4 years of Ruby experience, that was the best tool for the job since I'd get the most productivity out of it. To date, I don't regret this decision. There are some things Ruby is bad at, some things it is good at, but I think I can overcome the bad with time.
2. I think the difference is that Vagrant is a tool focused on user experience and workflow. i.e. it'd be possibly to build Vagrant on top of libvirt. I think libvirt is technically stronger than Vagrant, but Vagrant provides a better overall experience. The overlap is small, we both have a lot to gain from working with each other.
3. A handful of people keep saying that Vagrant is just a "VM setup + provisioning" tool. Vagrant does quite a lot more. The main example I always use is networking because it is usually the most complex. Vagrant makes networks work across Linux, Mac OS X, and Windows, and sets up the hosts and guests properly. VeeWee doesn't do this and won't do this. VeeWee at its core was built to be a VM image creation tool. VM lifecycle control was bolted on later. VeeWee filled a major gap in the Vagrant ecosystem for a long time.
I'm good friends with the creator of VeeWee and we talked all the way back in Oct, 2011 of merging the functionality. He was all for it. Its almost been a year but this work is finally going to get started.
I don't want to talk too much about it until I really start coding, but I can say that no one in his thread has really seen the true scope of what I'm trying to build here. They will. :)
Yes, the project seems to have around 380 files. I'd be interested to hear from the author if he's reading this thread why the codebase is at this size. Perhaps VirtualBox quirks necessitate a large part of it?
Still a lot of files but not 380, just wanted to point that out. Each individual file is actually rather small. There are only a few larger files in there (Vagrant::Environment actually perhaps being the only one).
"Why the codebase is at this size." Well, Vagrant started very small, but in the 3 years of its existence, it has gained a lot of what I've previously heard called as "code wisdom:"
* Generic host support since some tasks are host-specific. For example, setting up NFS on Mac OS X differs from Ubuntu which differs from FreeBSD. Vagrant knows this.
* Generic guest support. Setting up a variety of things is guest OS specific, so again, Vagrant knows how to configure networks, mount folders, etc. on a variety of different guest operating systems.
* VirtualBox bugs. Vagrant has quite a bit of code to work around VirtualBox bugs.
* Networking is hard. Vagrant does quite a lot with networking to ensure your networking works as expected. It also does things like port collision detection to verify that any ports you choose won't collide with any other VMs as well as making a best effort to check if that port is open, and erroring in that case as well.
* Plugin interface. Almost every part of Vagrant at this point in git master is customizable. This isn't overkill, this was built out of seeing actual human need for this. This lowers the number of features I need in core and actually fights being overkill, because more things can be built with plugins.
* Communication abstraction. Linux and POSIX-like guests with SSH is great, but what happens when you want to run Windows guests? SSH CAN work but isn't ideal. For this reason, Vagrant is started to abstract out communication so that Windows can use things like WinRM, completely transparently. Also, if you use something like VMWare, it uses the VMWare guest communication API, which is blazingly faster than SSH. Again, this is all transparent to plugin developers, provisioners, etc. It "just works."
It does much more, but I think you get the idea.
Building something that "just works" comes at a cost. And that cost is usually complexity. Can you build a very basic tool to control VirtualBox virtual machines in a small shell script? You betcha! Vagrant started this way. Can you make that reusable? Yeah, but prepare to add some complexity. Then make it run on every major OS and support some nifty features and you got yourself a big project.
Vagrant does have support for the general case, where something in 7 lines will be a bit more specific. For example, Vagrant will handle cross distro/OS NFS shared folders.
I'm not sure what are the benefits of supporting many VM providers for a tool that is meant for development (as far as I understand at least). Isn't VirtualBox sufficient? It is open source and runs on Linux, OS X and MS Windows. Perhaps I am missing some core idea.
Then there is veewee[1] which started life as a Vagrant plugin but as of version 0.3.0 it can be used standalone and one can ssh to the boxes using it, just like with Vagrant. If they were to add provisioning it would probably be an alternative to Vagrant feature for feature so maybe we're seeing a bit of competition between these two tools there.
On Ubuntu there is VMBuilder[2] written in Python but it doesn't offer provisioning as far as I know.
Last, as others have mentioned there is libvirt[3] which offers tools for communicating with different VMs but I don't think it offers a tool that matches what Vagrant does 1:1 (most notably the provisioning part).
The power in Vagrant is not the technology, it is actually the workflow. Vagrant is now in use by thousands of companies worldwide (some of the bigger ones you can see on vagrantup.com), and they consistently tell me that the power in Vagrant is that every developer, designer, etc now knows they can get some code and just type `vagrant up` and be working within a few minutes. When they're done? `vagrant destroy`. Small example, but Vagrant provides a consistent workflow.
There is a benefit to having this workflow on other systems.
First, VirtualBox has major problems. VirtualBox routinely kernel panics on a variety of operating systems, even leading to the point that there is a commit to the linux kernel that ignores crash reports from VirtualBox.
Second, the whole goal of Vagrant is to have development mimic production as closely as possible, and to make development fun. Some companies that use Vagrant can't mimic production very well on a single laptop, so they have a private cloud. They still want to use Vagrant -- because of the workflow! -- and this caters directly to them.
Also, for people on Linux, I know its not full virtualization, but some problems can be solved with simple LXC. Those people write scripts around LXC because LXC is pretty simple, but it would be pretty nice to get the benefits of Vagrant there, too. That is coming! Vagrant will support LXC.
Libvirt doesn't compare to Vagrant at all. If anything, they compliment each other.
And veewee-like functionality will make its way into Vagrant core very soon.
I had so bad experiences with Virtualbox (Dropping interfaces, crashing, ...) and switched to VMWare (Mac OS X). So I applaud the support of different providers.
1. https://github.com/automaticmode/active_workflow 2. https://github.com/automaticmode/active_workflow/blob/master...