TLDR; The idea of the new system is to turn Apple’s existing network of iPhones into a massive crowdsourced location tracking system. Every active iPhone will continuously monitor for BLE beacon messages that might be coming from a lost device. When it picks up one of these signals, the participating phone tags the data with its own current GPS location; then it sends the whole package up to Apple’s servers.
I'm a member of this project. We are trying out few new ideas and your feedback and comments would be awesome! If you have been using TensorBoard, Visdom etc then you would probably find this work useful. Please let us know any questions you have.
TLDR; AirSim is simulation software built upon Unreal Engine 4 from Microsoft Research. It now includes car simulation and real city-like environment where you can test your self-driving algorithms. You can just download the binaries and start calling its Python APIs in minutes.
Want to work on AI + Robotics at Microsoft Research, Redmond, WA? Does Aerial robotics, Unreal engine, physics, building low level firmwares excites you? You will need to have demonstrable great C++ coding chops, passion for tinkering, ability to get into unknown areas fast and great CS algorithms design skills. Any experience with Unreal Engine, Unity, Physics engines, robotics, drones, firmwares, deep learning etc would be big plus as well. We have already shipped AirSim[1] and we are excited to take it to next level and advance state of the art in reinforcement learning among other things!
The position requires US work authorization and interviews would be in-person at Redmond, WA. Please send resumes at shitals@micro####.com.
Want to work on AirSim [1] as contractor at Microsoft Research, Redmond, WA? Does AI, Aerial robotics and drones excites you? You will need to have demonstrable great C++ coding chops, passion for tinkering, ability to get in to unknown areas fast and great CS algorithms design skills. Any experience with Unreal Engine, Unity, Physics engines, robotics, drones, deep learning etc would be big plus as well. Send resumes at shitals@micro####.com.
Can this be used for more than just multirotor drones? IE - what about other drone aircraft, as well as ground and water-based craft?
I'm currently working thru the Udacity Self-Driving Car Engineer nanodegree; as a part of the first term one of the projects used a Unity3D-based simulator to train a CNN to drive a vehicle.
It would be nice to have a more generalized simulator for this kind of work, to be able to simulate potentially more accurate environments (off road, especially). I could also see interesting work being allowed for underwater ROV environments as well...
As it is, I have considered (for an off-road and more general use) re-tasking TORCS or Trigger Rally (though after just doing some research, it seems like there are ton of these simulators - so maybe one more isn't necessary).
Still - I'm curious whether other environments and vehicles are being considered...
EDIT: It looks like Udacity recently open-sourced their simulator (one thing I have noticed about this course is that sometimes they do things, but don't let the students know about it - that, or maybe I've just been too focused on my work to notice, which is a very real possibility!)
You are absolutely at the right place. We have gone to extra length to design our code so that vehicle specifics are abstracted away behind interfaces. Also things are easily switchable. For example, don't like our physics engine, bring your own! As mentioned in release notes, we are working on other vehicle types. However water based crafts are not on our radar. My guess would be that physics would be more expensive and less accurate in general for water based vehicles.
Is the benefit of making it a separate tool related to making better/more APIs or avoiding graphical shortcuts made in video games? Or other benefits?
I realize that Microsoft wouldn't have access to GTA5 source code, but Microsoft Studios could have other games, that have already made substantial work in terms of graphical representation and performance, thus saving time for more API work, if the platform could be reused.
One of our major focus is to use AI in real world. A lot has been done in applying AI to video games but we are still taking baby steps in applying it in real world. The question that we want to answer is this: can we train and test algorithms in simulated world and then use them in real world? This is known as transfer learning problem and its a huge area of research where people are investigating, for example, how can we use 80% simulated data and 20% real world data? The games typically found on commercial market are created for entertainment purposes. For example, game creator might decide to cheat on physics to optimize performance and may configure things so its easy but not too easy. Our focus was rather to pursue good physics, sensor models, vehicle dynamics, hardware-in-loop and so on. Also, we wanted to make this project as open source. Most commercial products typically involve complex chain of 3rd party licenses that almost make it impossible for them to be open source.
No questions, just a big thanks for doing this, and even more thanks for releasing the source code. Also, thanks for doing it in UE4. This could be very useful to me where I work.
As someone who works with autonomy for UAVs, previous simulation tools such as gazebo etc. are great for developing and testing control and path planning algos. However, developing, testing and tuning perception techniques such as visual inertial odometry, vision based slam remains and challenge since it typically requires getting your hands on real world data. Photorealistic rendering envs should make it a lot easier for developing good perception algorithms faster.
The tools looks very promising, thanks for sharing it and keep up the good work.
1. Do you have plans to adapt this to Autonomous Cars anytime in the near future as a potential temporary work-around for navigation / technology shortcomings?
2. Why not port this in partnership with Forza to make race tracks out of real world environments? I had an idea along this line, though it might be way out of scope heh:
When the readme says that the official builds for Linux will arrive in a few weeks, does that mean that it is not yet possible to compile from source also on Linux or is that already possible?
We are internally able to compile the core (AirLib) however there are few rough edges and also we want to make sure Unreal also works as expected on Linux with our plugin.
Are there plans for implementing weather simulation? Things that might force course adjustments or reduce visibility would be a useful addon to the simulation. I know one team when I was in school would have loved something like that when they were making a drone meant to scout forest fires and navigate through smoke.
This would be very important for lighter than air drones. Which... might have much more commerical application anyways due to their long loiter capability.
I have been scanning through the repo for the past hour and I am blown away. Amazing work! As a recent CS graduate who's dream is to work on drones, virtualization and autonomy, is there any place I can shoot you a resume?
We are not officially supporting Mac yet, but contributions are welcome :). Our code is cross platform and given enough efforts, I think it can be made available on pretty much any platform. One thing to think about is GPU requirements. Even moderate environments in Unreal can easily require GPUs with at least 4GB RAM.
Many students of the Udacity Nanodegree have found out the hard way that using a stock MBP for CNN training isn't exactly great. Even MBPs with the NVidia 750m struggle (my home workstation is Ubuntu 14.04 LTS with a 750ti SC - and it is only barely adequate).
Honestly, I can't see how you'd be able to run a high-quality 3D sim while training a CNN or other ML model at the same time without something beefier (I haven't tried it yet with my 750ti - but with only 2 gig I have a feeling it wouldn't like it much; it's ok running with the trained model, though).
What students usually (I think) do when all they have is an MBP (especially one with only intel graphics) is train on an AWS GPU instance, then run the trained model locally. This class has been instructive for me in showing that for anything beyond the toy stuff we're playing around with, you really need a beefy ($$$) system. I wish I had the kind of money needed in order to do so (an 8-way 1080 or TitanX box would be sweet - fat luck with me getting it past my wife, though!).
Awesome work! Wondering if it is possible to get data (telemetry, IMU, etc) from a drone during simulation? And is it possible to have multiple drones in the simulation?
Yes, we are supporting to get telemetry as well as ground truth. We also support multiple vehicles but you will need to then build your controller for them or set up multiple HITL/SITL flight controllers.
That's awesome! Can I shoot you an email for more information? I'm doing some research where we're trying to improve flight time with IMUs (GPS-denied environments) and AirSim looks like a great environment for us to get more data and train our algorithms!
Yes! We have been fortunate to find few in-house drone racers and learning the grit behind their skills. Getting SITL/HITL with Betaflight and/or Cleanflight is definitely one of our important goal in coming months.
Suppose someone found a great algorithm for drone control based on your simulator and decided to mount a gun on the drone and kill a whole bunch of people.
With all the hotness in AI, rich simulations and algorithms, what steps are you taking to ensure power doesn't fall into the wrong hands?
Very cool, have you measured the model using real flight data and seeing the discrepancy between the model generated photo/sensor data and the real photo/sensor data? And is their a plan for laser tag mode where multiple vehicles compete with each other for air dominance :)
This is not Robotics Studio. It's a simulator for training and testing deep learning/reinforcement learning algorithms for autonomous vehicles. However we are sure community will find other uses :).
No, we are not OpenAI competitors! In fact, we had very much like to work with OpenAI and think about how do we integrate together to advance the research.
