Uncloaking Orion: Behind the Scenes with Dan Phung


From studying fruit flies, to getting fired from The Strand, to working at Popeye’s Chicken and Biscuits, Orion Software Engineer and resident data diver Dan Phung is all about the hustle. He recently gave a talk and interview at the Splunk .conf 2015, titled “Reaching for the Stars by Splunking Orion: A Distributed Near Real-Time Group Communications Platform”. Here he tells us a bit about how he ended up at Orion and what makes him go.

How did you end up at Orion?

One random day while walking around South Park after lunch, I ran into Andy Isaacson in the neighborhood. We had worked together at VMWare so I asked him what he was doing now. He told me he was working at a startup called OnBeep that was still in stealth mode. I looked up the website and it showed a bunch of things associated with space, like space shuttles and astronauts, and vaguely described something about group communication, which seemed pretty awesome. There was a link on the site to open positions that led to Angel List. The only position open for software related things was DevOps, which was a little out of my experience but I applied anyway.

I was originally just interested in finding out what they were up to. That led to an initial meeting over coffee with Greg Albrecht, one of the founders of the company. He realized that I was a software engineer, so he asked me to come talk to the software team and that led to me joining the company.

I had already been looking around for the next step in my career and I knew I wanted to join a startup where I could really have a big impact. I like the hustle and uncomfortableness of learning new things and being challenged to do something that I’m not used to doing. I want to hone the skill of being resourceful, learning and applying concepts quickly, and doing these things in an environment with limited resources and being mindful of cross-cutting priorities across the company. That’s what I tried to do at VMware but had less impact, because there was more bureaucracy in the way. But this ethos is evident in the way I got into grad school, the papers I’ve written — in neuroscience, distributed computing, e-learning, virtualization — are all over the place, because I love learning and getting my hands dirty.

What really drew me to Orion (formerly OnBeep) was the compelling product of voice-based group communication. The paradigm of group communication and voice streaming over a device is a very simple idea but it’s really hard to get the entire user experience right and I really want to help bring this vision forward. I want to learn what the challenges are for getting a product from inception to global acceptance — similar to Google or Twitter, but with a new spin on the voice network.

It sounds like there’s a story in there about grad school…

The story is… I have my undergrad degree in biology and chemistry, which is where I dissected the nervous systems of fruit fly larvae and did some research in cognitive neuroscience. I had wanted to be a geneticist but the life cycle of biological research was so tedious and boring — spending night after night watching fruit flies mate in the lab, feeding their spawn, taking notes on the fluorescent expression of a particular gene and extrapolating that data to animal behavior, all to write one paper — just wasn’t doing it for me. I then switched over to neuroscience and analyzing MEG data using ICA techniques and got a couple papers out of that, but helping professors and grad students with research in undergrad quickly got me burnt out.

After graduating I got job at a bookstore called Page 1 in Albuquerque (where I grew up) and that was really fun! Waking up at 6 o’clock in the morning, biking to work in the dark, getting coffee, and chilling with books – I loved the zen of it. I started off at the cash register and shelving books. At some point I fixed something with a computer and they found out that I knew a little something about computers and sent me to the IT guy.

I helped migrate the entire bookstore from Windows desktops and a Microsoft database and cataloging system to an open source MySQL database and Linux-based desktops. From installing the desktop software, helping train their staff on Linux applications, and writing Perl migration scripts and database tests, I did a lot that summer and I think I did it for minimum wage! Then the summer was over and I needed to get a real job.

I was looking at grad schools and thinking New York has this great nucleus of stuff going on — Columbia, NYU, Cornell, a bunch of schools had computer science, neuroscience, good biology… I had no idea what I wanted to do. So I packed up and went out to New York, but couldn’t find a job since this was right after 9/11.

I got my first job at The Strand and got fired for wanting to organize them. Nothing there is organized. You can’t even look up a book thinking the authors are alphabetized. I think the idea was that you wander around and discover new books. I was like, “hey, I just did this thing at this other bookstore where I organized it and set up a database…” and they just shook their head and said “I don’t think you’re going to fit in here,” and that was it.

So, desperate for a job and wanting the fringe benefits, I started working at Popeye’s Fried Chicken, where I learned to speak a little Spanish. In the meantime, I was out door knocking at labs at Columbia and taking some classes.

I got into a lab and asked if they have any volunteer positions. I learned Java that summer while finishing 2 summer projects. At the end of the summer the professor, who happened to be head of the PhD committee, asked if I wanted a GSAS (Graduate School of Arts and Sciences) research assistantship. I hadn’t even applied for the Master’s program yet, so she said to get on it. That weekend I took the GRE and applied, and that’s how I got into Columbia University.

What’s your role at Orion?

Officially my role is Software Engineer but underneath systems engineering is really what I’m doing. My strength is being able to understand the larger picture of how things should work and dive down quickly and efficiently to attack different aspects of the problem. For example, a given week can include writing and debugging backend server code, jumping into firmware code and then back out to writing a data-backed report on how customers use our network. I’m currently focused on helping the Firmware team but I’ve spent much of my time looking at data and metrics from different aspects and working on the Services team.

What does a typical day look like for you?

Currently because I’m on the Firmware team, I’m helping out in whatever way I can. I’m helping to increase communication from the firmware team out to the rest of the software team so that we can be more tightly aligned in terms of our planning. I also see that role as buffering much of the daily interruptions that can happen so that the main firmware developers can concentrate on shipping code. To that end, I’m checking the previous day’s commits in the morning and decide whether to cut a release. If there are significant changes that affect the rest of the software team, then I distribute those changes to their devices. I then check to see if I can unblock anybody on the team and then head off to work on tasks that are needed for firmware that I can directly contribute to.

Is there anything about the job that you didn’t expect?

I didn’t expect to be writing things like the Privacy Policy and the social contract documents but it was very interesting exploring these things for our company and funneling the ideas that people have. People here at Orion are really passionate about the privacy and security of our customers along with very socially progressive ideas like diversity and inclusion in hiring. Our company has a lot of very cool people that want to do good.

What do you like most about working at Orion?

I love being able to hustle again and have an impact. I love being able to learn and do new things while developing some of the craft I already know. I get to dive into data and learn insights into how people are using our network. I get to learn golang and build a service that’s deployed today while helping our team get a critical piece of our infrastructure more maintainable. I get to go to Las Vegas, give a talk and sell the idea to other salesmen, directors, and VPs. This has been a pretty fun ride so far and the people here really make it enjoyable. Everyone’s really intelligent and willing to listen and work hard at bringing this vision forward.

What’s been the most challenging?

Keeping focus & prioritizing. There’s so much to do and we don’t have enough people to do it all. There’s a challenge to understanding what things to do that are the most impactful for us and weighing those decisions with their impact in the short, medium and long term.


What do you like to do outside of work?

Play with my babies, go biking, running, brewing beer, drinking beer. I love food; food is so good. I cook a lot.

I cook pho and a bunch of other Vietnamese foods. I can also make the perfect roasted chicken and, as at least 25 other Orioners will testify, I make a mean bowl of Posole. I started at Orion around Thanksgiving and we had a holiday party potluck that week. Wanting to come in with a good impression I made New Mexican Posole, which seemed to go down well.

What’s your favorite travel destination?

My favorite travel destination that I’ve been to is the national park system in southern Utah – Zion, Bryce Canyon, Capitol Reef, Arches and Canyonlands. I love hiking around such majestic natural beauty.

The travel destination that is on the top of my list to go to is Vietnam, which sadly, I have not been to.

How do you use Onyx & Orion?

I mainly use it with Julie (my wife) on the weekends as we’re herding our sheep (kids) across town. It could be in the house or getting out of the house when we don’t want to yell at each other or text ‘cause we’re grabbing our kids out of imminent danger. Recently we were at a birthday party on Stow Lake and I was in the middle of the lake on a boat with 4 kids. One of the kids’ moms asked Julie to text me, “The magician is about to come; can you ask how much longer he’s going to be?” Without breaking eye contact, Julie presses the button [on Onyx] and says, “The magician is about to come. How far out are you?” Me: “We’re about 50 yards away.”

I’ve also used it to reduce my “yell-to-talk” messaging technique where I have to yell across the apartment to ask where the diapers are. Usually this technique doesn’t work and I end up having to run to the other room. Using Orion and Onyx across rooms helps with this a lot.


What are you most excited about for the future of Orion?

I want to help drive us towards this vision where Orion is the communications platform for group communication. It’s exciting to think about a world where there is a critical mass with everybody using Orion and we can have a massive online group conversation. There will still be a role for discrete communication through text, like messaging apps or SMS, but voice is both more instant and expressive. I also can’t wait to see what our awesome hardware and design team comes up with for future accessories. I can imagine buying and sewing on a button or pin to my jacket that connects me to the Orion network and I use voice commands or the phone app to tailor my voice communications feed.

I also want to understand the answer to questions like “How do you get people to equate the Onyx as the thing for group voice communication?” I want some kind of tagline like “Be On It – Orion Onyx!” so that people know to have the accessory paired and in a group all the time. I already do this on the weekends with Julie. We know at some point during the day things will be too crazy (with two toddlers) to make a phone call or text each other as a form of communication, so being “On” the Orion network gives us an always-ready form of communication.

This weekend I was thinking of talking trash to my friends about some football games. I just wanted them to be on Orion so I could do it quickly. I didn’t want to text or Tweet them. I needed to get the trash talking done right after a play was made and with my voice, which is so much more expressive compared to texting or hashtags.

These are only some of the benefits I hope we can provide at Orion. The ideas that excite me are things like how people will use Orion in an ongoing global conversation about a current event or what cool new unknown venues of communication can this paradigm empower.