Ranger's HUD Tutorial (v0.1)

What's a HUD worth to you?

  • < $25

    Votes: 1 7.7%
  • < $50

    Votes: 4 30.8%
  • < $100

    Votes: 4 30.8%
  • < $250

    Votes: 2 15.4%
  • < $500

    Votes: 2 15.4%

  • Total voters
    13
Status
Not open for further replies.

RandomRanger

DMO, Armory, and Forum Support
Division Staff
Community Staff
Member DIN
S063
Ranger's HUD Tutorial v0.1
Parts
Note
: Not all parts are required for all features. If you want a quote for how much a certain feature should cost, ask in the thread and I'll do my best for you. There's also a fair amount of wiggle room as far as how much a specific component costs, with currently the only exception being the display. You can also send me a desired budget, and I'll scour the internet's for you to see how many features I can pack in for that, given I understand how to move the parts around and what's really necessary from them.

PartCostFeatures it enables
Raspberry Pi$35Required for all features
Battery bank$xxxPower
Some sort of speaker$5-XXXMusic, audio out, comms, etc.
Aux extender$5Plug in speakers
VuFine Wearable Display$200HUD, Games, Discord
Keyboard$16All features
Mics2x $9 ($18)Voice comms, PA system, external listening
USB audio adapter2x $9 ($18)Necessary for mic in
WiFi Adapter$10-25Internet access
Expanded usb ports$10May be optional, add more usb ports
Build-a-headphone-jack-part$5Helps plug speakers in
Mount Vufine Display$6Mounts the screen to helmet
Jumper wires$6Helps install speakers
Raspi Case$10-25Helps install raspi into helmet
Total$75-400 (ouch)

Overview:
This tutorial will cover how to install a computer into your helmet and get the following features working (with no promises of it being pretty)
  • Music
  • Voice Comms (Discord)
  • Game streaming
Music is provided in two forms: YouTube (which requires little to no effort to setup), and Raspotify which is a 'headless' Spotify client
Voice Comms are provided via Discord and the use of the web browser client (requires some mildly hacky stuff to get it working)
GameStreaming is done via the application Moonlight, which requires a PC with a nVidia graphics card (GTX 650 or higher)

Basic Setup:
Installing the Raspberry Pi:
I don't want to cover how to setup/install the OS on a raspberry pi, as it's beyond scope and has been covered many times elsewhere. Here's their official directions: https://projects.raspberrypi.org/en/projects/raspberry-pi-setting-up/2
Here's an overview of installing the os:
We're going to use Raspbian, fyi. Download their NOOBS image, and extract it onto a empty sd card, then put it in the pi, boot up, follow the directions on screen.

The part I need to cover is installing the Pi into your helmet. I used two things for this, one I had a case on my Pi which allowed me to affix velcro to it, and two, I used velcro on it. I am wearing a gen 2 Mk VI helmet, which has a nice slot of empty-space sun-blocking-visor above the... gold-see-through-visor (we need more words for this). Here's what it looks like, I simply velcro'd the pi into this space and it has been working well.
View attachment 272250
Hopefully that's an easy step, I didn't struggle with it. Interestingly enough I didn't have an issue with the top of my head hitting the pi and becoming uncomfortable and pained.

Installing the power supply:
Next we need to install the power supply. I didn't list a reccomended power supply frankly because there's so many of them and people have widely different budgets. What I recommend looking for in a power supply though is 2 output usb ports (1 for pi, 1 for charging the display if you go for the screen), and that the usb ports are on the end and not on the sides. One of the issues my battery pack has is it with the usb ports on the side, installing it the way I did puts a lot of strain on the USB plug, and I broke one of my cables over time. My helmet came with foam on the inside, and I simply velcro'd it to the back of that foam which has worked well for me. If you don't intend on using the battery outside of the helmet you could use hot glue. I do recommend making sure you have some sort of barrier between the back of your skull and the battery pack because otherwise it would be quite uncomfortable. I also found the foam quite useful for cable management and hiding the cables, though this could also be done with tape if that's not an option for you.
View attachment 272251View attachment 272252

Installing the speakers:
First off, what speakers should you buy? It doesn't really matter too much, personally I had a busted bluetooth speaker lying around that I gutted apart and stole the drivers out of. An important thing to note while picking speakers is to make sure they either have female pin connections, or long wires (like 12in), as this will help with the wiring. Don't buy any that don't have wires connected... cause idk what to do then. If you can find speakers with long wires then you don't need to buy the jumper wires listed above.
This one I had a little bit of difficulty with. In order for me to get the speakers installed, I had to first cut holes into the side-foam and put them there:
View attachment 272253
This should be relatively simple, however one of my speakers refused to work with hot glue and I had to resort to gorilla glue (the other one affixed fine). One thing to note while installing the speakers is to make sure that you glue them down in a manner such that the wires attached to them are accessible.
Wiring the speakers:
This is where we use that heaphone jack adapter thing. Using that we are able to connect wires into it and create a headphone jack specifically for the speakers in your helmet.
View attachment 272254View attachment 272255View attachment 272256
Using that jack, you simply need to connect the wires to the speakers. The headphone jack has labels for L (left speaker) R (right speaker) and ground. In my case, my wires were short so I needed to use jumpers to stretch them to the jack. To do so, I simply plugged a jumper wire into both of the female pinholes and then connected the other wire of the jumper to ground and either L or R depending on what I was plugging in. It is worth noting here that after installing wires here you will have 1 wire in L, 1 wire in R, and 2 wires in ground. It is also worth noting that it does not matter which wire on the speaker you connect L/R and ground to, so long as you connect both L or R and ground. It will work either way.
This isn't necessarily what jumper wires are meant for, so it may be worth taping them onto the speaker connector if they feel like they'll fall out.

If you're using long wires instead of a jumper connection, simply strip the end of the wire (about half an inch) and stick it into the headphone jack's slots, then screw it down. This is a more elegant solution, but my wires weren't long enough.

Once all that is done simply connect the headphone jack to the raspberry pi with the aux cord. If you bought a mic + usb audio adapter, you will want to plug the aux into this as the audio quality is better. Note that in order for sound to come out of the USB adapter instead of the headphone jack you will need to tell the Pi to do so in the settings. How to do this is shown in the mic setup.

If your using a usb sound adapter, I recommend just finding a way to stash it on the side of the helmet using velcro if need be. I was able to get away with simply stuffing it into the side-foam. You can't see it, but it's just sitting right behind this foam:
View attachment 272257

Installing the internal mic:
This was fairly simple for me, I taped the mic to the bottom of the helmet so that the mic head was pointed at/close to my mouth, and then plugged it in to the USB adapter. You can't see the mic connecting to the adapter because it is behind my side-foam.
View attachment 272258

Configuring the mic/speakers:
This is all done on the Raspberry Pi. I recommend plugging it into a monitor/TV to work on it, with a full size keyboard/mouse.
First off, sorry for the pictures of a screen. I normally do high quality screenshots, but that's more difficult on a raspberry pi.
So here's the steps, open audio device settings;
View attachment 272259View attachment 272260View attachment 272261
In device settings, select the USB Audio Device from the dropdown (there will be two if you got two of them) and click Make Default.
View attachment 272262
In order to see the settings I do, click on Select Controls... and check all boxes.
View attachment 272263
Now your audio and mic should be routed through that usb card. At the time of writing, I have not yet gotten functionality out of the second microphone. If you're playing some audio (via youtube or something, covered later) and not hearing anything, try rebooting. Sometimes open applications don't detect the audio device change.

If your mic is quiet, you can increase it's volume on the capture tab in this app.

Getting internet access:
If you bought a wifi card that was advertised with compatibility for the pi then you should be able to plug it in and go. Otherwise, if you bought the one I did you have a 2/3 chance of needing to install drivers for it. In order to do that, follow the instructions found on this site: (UPDATE) Drivers for TL-WN725N V2 - 3.6.11+ -> 4.xx.xx+ - Raspberry Pi Forums

Also I have made a script which will automatically install the right driver version if it is available. The script can be downloaded using
Code: Select all
sudo wget http://downloads.fars-robotics.net/wifi-drivers/install-wifi -O /usr/bin/install-wifi
sudo chmod +x /usr/bin/install-wifi
Use command sudo install-wifi -h for details on how you can use it to download or update the wifi driver. This script will automatically determine the wifi you are using and search for the correct driver to download for the kernel version you are using

That should be it for this. At this point you should now be able to use YouTube using the built in webrowser.

Getting Discord to work:
This is all done on the Raspberry Pi. I recommend plugging it into a monitor/TV to work on it, with a full size keyboard/mouse.
To use discord, we will use the browser version. Unfortunately, the Discord browser does not support the browser we're using and so we have to trick it. First install the chrome web app Chrome User Spoofer. I used that one, but there are many. Once that is installed, use it to switch your user to IE 10.
View attachment 272264
After this is done, you should be able to use discord via their web app. If you have difficulties getting the mic to work, make sure you have the mic plugged into a usb adapter, and that the usb adapter is set as default in the pi's audio settings. Also make sure that chromium (the browser you're using) has given the site permission for the microphone, this can be done in the browsers settings.

Preparing the display:
This one is tricky and risky, so don't go too quickly.
Here's the hard truth, this display costs $200 (for whatever reason), and we need to smash it. Let me explain. This display works by having a tiny display shine through a lens and into your eye, and because it was intended to be used on a pair of glasses it comes in a fixed plastic case. There's one problem with this, if we leave it in that case it doesn't have the flexibility we need to maneavor it around in the helmet, and leave room for our heads. See here:
View attachment 272267
That is an early version of my HUD. With the display setup like this it is incredibly difficult to get the helmet on/off without breaking anything (including your eyeball) as the display sticks forward and that black plastic arm is in the middle of the helmet. Fortunately someone else had done a teardown of the display: and I was able to see how the insides looked . From this I concluded the only natural solution to an expensive and fragile display not fitting within a tight space that's being worn, removed, occupied, and breathed into, was to start smashing things.
Note this image:
View attachment 272268
Here we can see that the black panel on the inside comes off. I was able to do this without too much difficulty (but I also have an iFixit Pro Toolkit), and you'll want to do that as well. After removing that panel, you'll notice the only thing holding the display on is a tiny plastic tab/bar near the display. See here (thanks for these images u/old_and_limitless)

View attachment 272269
We want to break that tab/bar thing off so we can slide the display and its flex cable out of the plastic enclosure. Be careful, but not shy. To break it off and open the panel I used my beloved iFixit spatula. Once that is done, break off the curved part of the plastic part of the enclosure. This is now useless and will only get in your way. See mine here:
View attachment 272270View attachment 272271

That, was the easy part.

Mounting the display:
This is the tricky part.
This is where we mount the display inside the helmet. Simple. However, there is one very critical complication here: the display, wherever it is, must be pointed directly at your eye. The smallest deviations can result in either a blurry image, or chunks of the display clipping off (being covered up and not visible). To get around this issue, we bought a mount that allows us to maneauver in 3dimensions. That's the odd screw-on-a-ball thing inside a black cylinder (example of the same hardware holding my Vive lighthouses up). There may be better solutions than this, but this is the best I found. The trick is being able to position something by rotating it in 3 dimentions.
View attachment 272272
I attempted another method using two hinges glued to each other, but it didn't work. The hinges were too loose and didn't hold their position.
View attachment 272273

Here's the important thing to know for this step: find a way to fit the mounting hardware, and the display, inside your helmet. For me, the best way that fit was sideways by my chin. If my helmet was shaped slightly differently, I could've also stood it up vertically behind the mouth piece. Just remember to leave room for the mic. Here's what mine looks like:
View attachment 272274
It may be difficult to see the mounting hardware because it's hidden behind black duct tape, but the display is hot glued to the top 'spinning piece' (intended for a normal use case, like a camera), and then the hardware is simply duct taped to the helmet. Duct tape has problems such as an increased amount of flex, giving me a hard time to keep my display at the correct angle. However, I did this because I had difficulty getting hot glue to bind the hardware to the helmet. I would recommend trying hot glue first, just be sure not to hot glue the bottom part of the black cylinder to the top part as these parts move to allow the tightening of the mount. From here you should be able to (carefully) maneauvour the display into the correct position to see. I did not say doing so would be easy, or comfortable, but it will be possible.

Once you have the display mounted, you can plug it in. Simply connect the display provided cable to the raspi and display, and connect the usb cable to the display and power bank (not technically necessary, but the in-display batter only lasts about 1.5hrs), and route your cables however you choose (I have them hidden behind foam).

If you run into an issue where the display isn't outputting what you expect, try rebooting and re plugging everything, and if that doesn't work disconnect the display from power, sometimes that causes an issue as well.

I do not recommend using this diplay to setup features of the pi, as it is tiny. Develop on a tv, use on the helmet.

Streaming Halo or any other xbox/PC game to the HUD:
This feature is actually a chain of other software features being used together.
Requirements: Xbox One, Windows 10 PC with a nVidia graphics card (GTX 650 or higher)

Note: I heavily, seriously, definitely, recommend having both the pc and the xbox connected to internet/home network using an Ethernet cable, and NOT over WiFi. For a use case like this, WiFi has a really hard time keeping up and is usually too inconsistent to be enjoyable. Yes, the raspberry pi is on wifi, and if you bought the card I bought it'll do ok, but we don't want the PC, the Pi, and the Xbox all on wifi. You have been warned.

Here's the overall process:

Stream the Xbox to the PC using the xbox's streaming feature.
Stream the PC to the raspberry pi using nvidia gamestream via 3rd party app Moonlight

How to do it:
To get xbox -> pc streaming working, follow this tutorial:
or another tutorial of your choosing. Other's have done plenty good jobs explaining how.
View attachment 272278

To get pc -> pi streaming, your pc must have nVidia GeForce Experience installed. If you don't have this installed already, shame on you, you've been wasting your graphics card away without utilizing it's fancy features. GeForce Experience is Your Gateway to Great PC Gaming
Once that's installed, make sure to enable shield game streaming from the settings. You may want to double check the settings menu if you are uncertain if your card supports streaming

View attachment 272275
View attachment 272276View attachment 272277
This game streaming is intended to be used with nVidia's shield devices... which I doubt you own... but we're going to use it anyways.

Once that is done, we need to setup Moonlight on your raspberry pi. I recommend doing this on a tv with a real keyboard/mouse plugged in.
Install Moonlight by following this page here (you are on the Stretch version of Raspbian): irtimmer/moonlight-embedded

Or here's a synopsis of what to do
Add this line to your /etc/apt/sources.list file for Raspbian Stretch based distributions


Run these commands:
wget http://archive.itimmer.nl/itimmer.gpg
sudo apt-key add itimmer.gpg
sudo apt-get update
sudo apt-get install moonlight-embedded

Once that is done, we must pair the raspberry pi with your pc. On your computer, open command promp by going to the start menu and type 'cmd' then hit enter. This should open a command prompt window, when it does run the command 'ipconfig' and look for a series of numbers next to the word 'ipv4'. I should be a number resembling '192.168.1.xxx'. For example mine is 192.168.1.236
View attachment 272279

Once you have that number, go back to your raspberry pi and run the command 'moonlight pair 192.168.1.xxx' replacing xxx with the numbers from the ipv4 address you just grabbed. It should output a 4digit code, go back to your PC and enter that code in the prompt that popped up in the lower right corner. If that did NOT happen, double check the ip address is correct, and if that's not it then I recommend googling troubleshooting with 'nvidia shield pair not working' or 'moonlight pair not working'.

Once you've done that, plug an xbox controller into your raspberry pi and run this command
moonlight stream -720 192.168.1.xxx
If you're using a usb audio adapter, run this command instead
moonlight stream -720 192.168.1.xxx -audio hw:1,0
If you have 2 usb audio adapters you may need to run this, depending on which adapter got position 1
moonlight stream -720 192.168.1.xxx -audio hw:2,0

After you run that, you should be able to stream your pc. Now simply connect your pc to the xbox and begin playing!

Disclaimer:
I wrote large portions of this from memory, and there may be some steps missing. If you attempt to replicate this and run into issues, throw me a message and I'll do my best to help you fix it. Chances are I had the same issue and already figured it out. I plan on reviewing and updating this as time goes on.
 
Very nice work, I guess as a fellow programmer I may one day have to take a crack at this... however for the poll is it how much would I pay for a hud or how much I think its worth?
I want to create more than just 1 version of the HUD to keep it accessible, so it's intended to help me figure out what types of budget zones I should target.
And yeah! Though, at the moment most of the interaction is via default raspbian features/gui.
 
Last edited:
I just bought parts to begin development on the worlds first in-visor halo hud, they should arrive friday. The kicker? It costs less than $15 and has the potential to pack way more features, and is easier (for you) to implement (all at the cost of more work on my end).
Details to come.


My previous HUD, while it meets all the check boxes, didn't quite satisfy me with its lack of an in-visor style. I'm here to shake things up once again.

Update: Holy crap, I might've just figured out how to play Halo IRL. This month is going to be amazing, and I offer no promises.

Probably should've done it this way all along. Darnit.
Also meant to put this in my other thread. Oh well.
 
Last edited:
Great write up! I was able to follow along pretty well. I'm definitely referring back to this guide when I finish my helmet. Really looking forward to your progress on your updated hud! Would the raspberry pi 4 be able to stream better than the pi 3? I'm curious to see if an increase in performance would make a difference.
 
Great write up! I was able to follow along pretty well. I'm definitely referring back to this guide when I finish my helmet. Really looking forward to your progress on your updated hud! Would the raspberry pi 4 be able to stream better than the pi 3? I'm curious to see if an increase in performance would make a difference.
Yes, the pi 4 would and I thought about picking it up. But quite honestly, I say wait at least a month and see what I come up with for the new HUD. I am optimistic about it to say the least.
 
I just bought parts to begin development on the worlds first in-visor halo hud, they should arrive friday. The kicker? It costs less than $15 and has the potential to pack way more features, and is easier (for you) to implement (all at the cost of more work on my end).
Details to come.


My previous HUD, while it meets all the check boxes, didn't quite satisfy me with its lack of an in-visor style. I'm here to shake things up once again.

Update: Holy crap, I might've just figured out how to play Halo IRL. This month is going to be amazing, and I offer no promises.

Probably should've done it this way all along. Darnit.
Also meant to put this in my other thread. Oh well.

He's not even looking back at that cool explosion! HE'S A HERO!
 
Here's a sneak peak at the new in-visor HUD version
273175
 
Oh, the horrors to your wallet of functional prop development. I'm very interested in getting the hud and other features into a Mark V helmet.
 
Hmm, I'm realizing that the minimap I have is a static image, and the one in the game has the radar sort of effect on it. If anyone know's how to make animations, I'm all ears.
 
I still say make one using a small screen and high magnification. It'll allow fitting them into the helmets much easier, but it will require more DIY and $$$
 
My Google Cardboards arrived today, so I finally got to see what the HUD actually looks like, here's the best I photos I could manage (the previous photo was a screenshot)
signal-2019-07-05-161319.jpg
signal-2019-07-05-161333.jpg
signal-2019-07-05-161442.jpg

signal-2019-07-05-161354.jpg
signal-2019-07-05-161415.jpg


I've noticed a few problems with the design.

1) The cardboard device is large as you can tell from the photo. With my current helmet, it takes up all the space my head normally would. This is obviously an issue, however based off some eyeball measurements if I could move my visor forward just a little bit, as well as made the front a little wider this wouldn't be an issue.
2) The view is fairly zoomed in. This is because I'm looking at the world through a smartphone camera, which doesn't have the same field of view as the human eye (go figure). This could be remedied by adding a wide angle lens in front of the device, but from my own testing it was possible for me to navigate around the house. In an environment like a con, where the majority of navigation is straight lines with left/right turns I don't think this would be too much of an issue, as you can use your neck to look around. Just don't take any tight corners (which you shouldn't be doing in a suit anyways).
Update: I just tested the HUD with a macro/wide angle lens and confirmed that this does indeed significantly reduce the zoomed in issue. However, an external lens takes up a lot of space in a direction we don't have. So... solutions.
3) The framerate is low, leading to a display that can be uncomfortable for people who are not used to it. Part of this is due to my 'upgrade'' to a Pixel 3a XL, which actually has half the graphical performance as my previous phone. I was able to reduce this issue by turning on a 'high performance' mode on my phone (my phone's rooted), but I imagine recent (2017+) flagship devices (Pixel 2, Samsung Galaxy 8, whatever iPhone was then) wouldn't have as much of an issue with this. Technically the display is limited to 60 Hz (60 fps), and normal VR systems run around 90 Hz (90 fps).

Cool things I noticed:

It's freaking cool to walk around with a Halo HUD, and Music (via spotify app and google assistant) and Discord (via app) still work so I haven't lost those features.


I'm currently working on getting youtube streaming to the HUD, as well as a media window (e.g. album art, current song, etc.)

Favor to ask
Would anyone be willing to put their phone into their helmet along side the visor (with the screen facing you, landscapewise) and tell me if you can still fit their head in? If so, you're helmet may be able to fit the hud. If not... well... definitely not.
 
Last edited:
Status
Not open for further replies.
Back
Top