Halo Augmented Reality System

So for those of you who don't know what augmented reality is, it's the usage of technology to superimpose a computer-generated image on a user's view of the real world, pretty damn cool.

I intend to create a pepakura Halo 5 Argus helmet, this is the easy bit (never thought I'd say that considering all the sanding involved), the hard part is the augmented reality system;
This will be done via coding a program on my desktop using Python (a coding language, I'll be using IDLE 3.2 for those interested) and a package called OpenCV (link here for those interested: http://opencv.org/),

I'll be coding a system that will overlay a Spartan GUI onto my view of reality using this software, for those really interested the nerdy bit goes into detail below, for those interested in pretty pictures I don't have many but look bit further down .

OpenCV is essentially pattern recognition software it can be used to recognise objects in images, easier said than done.... so I'll be interfacing this software with Python and putting the compiled (roughly as python is interpreted) code onto a Raspberry Pi 2,
this is essentially a micro computer small enough to be concealed in a helmet and with very low power consumption, due to the limitations of reality for a non-multi millionaire I can't technically create a true augmented reality system, instead I have to create and image to export into a virtual reality system, the conventional VR headsets cost a good few hundred pounds, so a no go there, instead I'm mounting a screen (of undermined as of yet nature) inside a phone VR sytem, currently I have in mind the FreeFly VR headset (https://www.freeflyvr.com/), sounds simple right..........

Fun Picture Section:
The Raspberry Pi computer to be mounted in the helmet:

The computer plugged in and turned on: (the module attached by the white ribbon cable is the camera)

A Halo inspired HUD overplayed onto a live video stream:


So the helmet,
for this helmet I planned to make an Argus helmet from Halo 5, partially because I like the design and partially because it has lots of empty space to hide electronics in. Unfortunately as far as I know of this helmet hasn't been unfolded yet, but that's no disappointment as I'll need to modify it anyway to accommodate all the gear I need to cram inside. For those who haven't seen the Argus helmet it's the one used by Linda-058 and it looks like this:


Anyway, all feedback is greatly appreciated, and happy costuming.

slick!

I built a HMD Years ago for a College project.
Based on the guts to a Xybernaut brand VGA display..... belt PC back in 06. take that google glass!

If you want a Off the rack system the Epison BT200 is VERY good . Ive got one on my list to buy for a project

have you thought of adding a GPS and waypoints. to other spartans? Look up APRS . It might be a nice thing to add.

Peter

I was actually thinking about this the other day. I saw a video a while ago of a guy that was working on a Scouter from DBZ that had the images popping up on the transparent lens. I was also thinking of looking into more information on Google Glass.
I need to get my first armor finished but this is exciting.

peterthethinker said:

slick!


I built a HMD Years ago for a College project.
Based on the guts to a Xybernaut brand VGA display..... belt PC back in 06. take that google glass!


If you want a Off the rack system the Epison BT200 is VERY good . Ive got one on my list to buy for a project


have you thought of adding a GPS and waypoints. to other spartans? Look up APRS . It might be a nice thing to add.


Peter

Click to expand...

Thanks, unfortunately my measly budget only stretches to a couple of hundred pounds a good 50 of which are already in the Raspberry Pi 2, ;( so no pre built glasses for me, instead I'm going to have to adapt a phone Virtual Reality headset to do this,


As for the GPS, I've had a brief think about that before, it wouldn't be to difficult with the compatibility of the Raspberry Pi, it could be quite fun to have a live time GPS attached to a world map/ local map in the corner where the map in Halo is,
thanks for the input,

DaMatt said:

I was actually thinking about this the other day. I saw a video a while ago of a guy that was working on a Scouter from DBZ that had the images popping up on the transparent lens. I was also thinking of looking into more information on Google Glass.
I need to get my first armor finished but this is exciting.

Click to expand...

Thanks I really appreciate the support, unfortunately this wont be a transparent lens, I'm hijacking a VR headset to make my system as I can't afford any of the pre-built transparent glasses.

Could you tie into a gps app on a cell phone like Friend Finder or something with the raspberry? Could the raspberry work in conjunction with a phone for things like text messages and stuff too? Would be cool to have a complete HUD system that doesn't require you to actually have access to your phone to keep track of things.

I believe it is possible to interface the raspberry pi with a sim card, so I couldn't completely replace a phone but it could hypothetically text, and maybe an interface through the phone could be possible but I'd need to find a way of interfacing a phone and raspberry pi, but it certainly could be fun to try. Maybe using a smart phones VPN system could allow communication between the two devices allowing an interface,

I have the same goal just without the programming skills that you have. My thought was a cannibalized pico projector top mounted.
I just recently got a 3D printer and am going to start working on a Reach Scout helmet which has room above the head and a overhang visor which would give me places to install components.
one of the few things that i wanted to install to go along with my display was pitch and roll readouts and distance with a low budget IR sensor (I think that's what I'm thinking of).

Love the display you have going currently. Can't wait to see more.

Looks great! Wow.

Thanks Mara,

Viridiana, sounds cool, the pitch shouldn't be too hard a small gyroscope should suffice, I thought about using projectors but I discounted it due to the offset putting them at an angle causes, as for an IR camera I may have to pinch that idea

Working on the raw obj file for the helmet now, so hopefully the helmet will be unfolded by the end of the weekend, I'll keep you posted, once again thanks for the great support and input.

The third-gen Pi features on-board WiFi and such - I'm surprised you haven't looked into it. I'm working on an ARGUS build myself, so I'd happily collaborate ideas with you and share files as I go.

Also, I have an unfolded base ARGUS helm file for your enjoyment - I used it myself, the scale is almost perfect for most heads. You'll need to work on the attachments, though.

Thanks, I did take a look at the third gen pi, but for me it seemed unnecessary, I don't really need the extra processing power (I hope ) and I have a wifi adaptor for my gen 2, thanks for the idea of collaboration that could be really good as I'm not the best at this I only occasionally dabble in 3d modelling and unfolding, my only problem is I need to alter the files so I can fit my electronics, so I may end up having to move the faceplate forwards slightly to fit the VR goggles in, anyway thanks for the help, I'll keep you updated on the modelling progress and contact you if I need some help,

Periscope the display!

Infact it will increase the optical path and only make it better to focus.
And it lets you use off the rack NTSC ( PAL in the UK) normal video off the PI's AV jack.
And it lets you use a transparent sheet for the lower mirror .

And now I see why your name is peterthethinker that's actually a really good idea that would allow a transparent overlay and prevent to much bulk in the front of the helmet, this is great and it would reduce the cost as only a mini projector would he needed that could as you say run off the HDMI output on the pi, thanks so much for this,

Take a look into Nuviz. It was a motorcycle HUD on kickstarter that got funded but eventually refunded (I was a backer ) but they use a type of periscoping as well from the bottom with a concave plastic to enhance.

Pitch and roll can be taken care of an accelerometer and if you want to get really advanced set the accelerometer to kill your shields and recharge them when it takes a significant shake.
I got ideas for days the issue it time and implementation. i need to get my helmet created first XD

Hey, just to make sure I understand this, do you plan to have a working visor (to view out of without the use of the AR projection, or would you rely on the AR projection? Either way, your idea is amazing and it's something that I hope to be able to do. Good luck with the project!

Oh! I just remembered this, for the radar take a look into Oakley's Airwave goggles, they have a neat HUD system that can display various things, like the location of friends, navigation, how fast you are moving, etc. (A fair warning this cost a pretty penny, about $650 U.S ,but it could give you some ideas if you want to look into how it works.)

Viridiana, that looked cool, I think I'm slowly moving away from the HUD periscope idea however, what I think I'm going for is modifying some virtual reality glasses, and you're right about the accelerometer, but damn, you really do have good ideas, you've got to be careful, I may end up pinching them all, nah but for mine, priorities are currently in getting facial and object recognition running,

Fabled, thanks for the support, you've somewhat got it, I'm not 100% sure myself though, it's all still in development, but essentially the idea for me currently is to have a camera feed going to my micro computer that will then do some processing and produce an augmented image, this image will then be transferred to a Virtual Reality headset, not a HUD, so unfortunately not completely true augmented reality but still pretty fun, and the goggles you're talking about are a tiny bit out of my price range

Anyway, update time;
I've been working on the 3D modelling for the helmet, so using the raw files extracted by Chernobyl I've taken them into blender and started deconstructing them to make a mesh that can be completely unfolded:
Complete:

Deconstructed:


However great revelations, I shouldn't need to mess with the complete design as I can use cinematic VR glasses for my display, so this should be pretty straightforward, for those of you who are hyper observant you will notice the cabling has been removed, this will probably be replaced once the helmet is complete by real cables, they may even be operational cables at this rate too,

Anyway, thanks for all the support it's great to see this level of support for such a strange project,

Hey man Pinch away. none of my ideas are things that i have created just thoughts that i have compiled so no reason you can't use them as your own. I'm probably going to be following this thread heavily so please post away. If i can ask one thing of you with that it would be share your code. you're producing with Python which is New and extremely marketable from what i know at the moment and i'm sure a lot of 405th Members would love to implement your ideas in the future. If you would like i will did up some of my old sketches and share with you the maps and details i had laid out in regards to design and implementation. I think the biggest issue you're going to run into is power which would require batteries or a battery cell which adds weight. Your helmet adds lots of options for space much differently than mind but be careful of that.

Fabled, I think the Airwave goggles use what's called Recon Instruments Live mod HUD which is a pretty sweet device. Something i've thought about integrating into my motorcycle helmet and they have the recon jet which is pretty much the sport edition of google glass. Another thing to look at but based on the route that he's going i'm guessing these aren't going to all in line with his needs.

Haha, thanks Viridiana, I'm more than happy to share the code once it's done, but unfortunately I can't ensure compatibility across platforms/ installations, but I can do my best to make the code as cross compatible as possible, the battery problem is fairly easy to solve, the raspberry pi is low power and can easily be powered by a couple of lipo batteries.
if you want we can turn this into a joint thread and you can post here too, but yeah the Argus has a lot of free space in the add ons, but for a normal helmet, a power unit in the torso could be a solution.
Design wise I should be ok, it's not that complex and I have some graphic design skills,
May I ask what helmet you are planning on mounting your unit in?

My Helmet is going to be Juns Halo Reach Scout Helmet. As much as i would love to make this a joint thread i don't have enough of my project going for me just yet. I am just getting set up with my 3d Printer and have yet to print anything, still trying to figure it out without anyone to help me at the moment but as soon as i do i will make a WIP Thread for my helmet and share it. For now i'm fine with helping you with ideas and inspiration! What VR googles are you going with? will they be permanently affixed to the helmet or something you put on and then the helmet?

Viridiana Sovari said:

My Helmet is going to be Juns Halo Reach Scout Helmet. As much as i would love to make this a joint thread i don't have enough of my project going for me just yet. I am just getting set up with my 3d Printer and have yet to print anything, still trying to figure it out without anyone to help me at the moment but as soon as i do i will make a WIP Thread for my helmet and share it. For now i'm fine with helping you with ideas and inspiration! What VR googles are you going with? will they be permanently affixed to the helmet or something you put on and then the helmet?

Click to expand...

Haha, nice, a 3D printer, you should be able to get some pretty good results, I'd love to see a link to that when your thread is up, I'm currently planning using these for my headset/goggles: http://www.ebay.co.uk/itm/271920962722?_trksid=p2060353.m1438.l2649&ssPageName=STRK:MEBIDX:IT , unfortunately not the best resolution but they will suffice for proof of concept and initial implementation, and I don't think I'll permanently fix the system in the helmet, instead I'll have it so it's removable as it'd be to much money for me to put into just one helmet.

Anyway, Update time:
So I've been quite busy over the last two days, a) setting up the code needed to create the HUD and b) working on the helmet itself,
so here are some photos of the deconstructed helmet:


Now to unfold the file in pepakura.............

So far on the code I've set up a custom coding environment to work in using anaconda (link here: https://www.continuum.io/why-anaconda) and OpenCV (link here: http://opencv.org/), this by default installs a development environment called Spyder which looks like this:

With the coding environment set up I can now get to work,
for those interested I'll also me detailing my code here to allow a reproduction of this in future, so here is the base script I made that overlays my drawn HUD onto a black image:


and the result:

Yes this is a very low res image, simply due to the display I'll eventually be outputting this on, in addition a lower resolution image is faster to process.
So the next task was to get this code to run on a video, the code to do this is shown below:

Unfortunately I don't have a YouTube account set up but I'll try and have one set up soon so I can demonstrate the full effect, but for now here's a sample frame:


All feedback and questions are more than welcome, hope you all enjoy.