I’m going to tell an honest story about product development and share some lessons learned. This is an age-old story of trying hard to hit the end-users’ expectations, missing, rattling around a bit, and finally succeeding. It is also a story as timeless as the “relationship” between engineers and operators.
Why on Earth, you may ask, would I share an “honest” story about product development? Though today’s corporate leaders are trained to maintain a pristine storefront, void of any blemishes, we all know there are two types of companies that have never failed at something. There are the companies that have never tried to do anything remarkable. And then there are the liars. I feel there is a lot to learn from mistakes in product development and I am happy to share ours because we are a better company now and we have a better product.
The story starts in 2014. Greensea started collaborating with STIDD Systems, Inc. (www.stiddmil.com) to develop a navigation system for their Diver Propulsion Device (DPD). The DPD is the most widely used mid-tier mobility system for combat swimmers, but it has traditionally used pretty rudimentary navigation. The goal of this collaboration, which is now more like a family, was to develop a really accurate navigation system for the DPD.
So, after a couple years of development and engineering, we started testing the first prototypes. We set aside almost 2 years to test the prototypes in open water because we knew the user experience was going to be critical and quite challenging. Developing a multi-modal inertial navigation system for a dynamic platform like the DPD, that could be operated by a diver while underway, is a complex problem and was certainly a new human machine interface concept for us.
We spent most of two years diving and perfecting the navigation system, which is now marketed as RNAV2. We focused on the accuracy, the sensor fusion, the functionality, and delivering the promised capability. Then, we shifted over to the user experience. We dove the RNAV2 in the DPD while applying all the best practices of UX design engineering. We instrumented the cockpit of the DPD with cameras, as well as the diver’s head, to capture the diver’s eye movements, hand movements, how many times he had to reposition his hands, and how many clicks it took to do a task. We timed task completion and task aptitude. We also developed a user satisfaction survey to help us quantitatively analyze our progress. We reviewed, revised, and deployed almost weekly during those years to arrive at a user interface that was simple, functional, and easy to use.
Then the big day came. We had a combat swimmer team evaluate RNAV2. We could see during the test that the navigation performance was dead-on accurate. High-fiving each other and slapping each other on the back, we were flabbergasted when the diver who was the pilot came out of the water and said, “Man, it works great – I wish I had two more hands to operate it though. It is pretty complex”. Dammit.
After years of testing the system for usability and engineering every possible button-push, knob-turn, and eye movement out of the user interface, we still came up short. So how did this happen? We are really smart engineers. We followed all of the established engineering methods. We EVEN used the damn thing ourselves. Where did it go wrong?
Here’s where it went wrong. None of us are combat swimmers. All of us on the RNAV2 design team were divers, and pretty experienced divers. Some of us have been commercial divers, some salvage divers, some recreational, and some military, but none of us have been combat divers. We have never piloted a DPD at night, in some part of the ocean where we probably shouldn’t be, while carrying weapons and rucksacks, trying to stay alive while delivering presents to the bad guys. We hadn’t actually stood in the operators’ shoes so we had no way to have their perspective. We tried, we got close, but we missed due to a lack of perspective.
The thing we missed was not really the RNAV2 interface, it was the overall task load on the operator. I think this is a pretty common problem when we try to modularly increase capability by just adding new technology. The combat swimmer was already completely task saturated (see above paragraph). Then we came along and gave him a computer, a screen, buttons to push, and information he had never had before. Though the interface was simple, it was enough to cause a real problem with the already task-saturated diver.
It would have been easy to roll over on the usability issue and push it to training. “With more training, you’ll get it” is a line I hear commonly when manufacturers are demonstrating their technology. But, that was definitely not the right answer in this case. RNAV2 offers a lot of benefits to combat diving operations. Plus, I wanted operators to love it, not just deal with it.
As the robotics industry integrates greater technology into traditionally manual processes to augment, assist, or even replace human operators, we have adopted a methodology of modular capability upgrades. In this model, we modularize capability as stand-alone products and add, not really integrate, those products into the human’s current process. What we miss is that the human is likely already task-saturated. We miss the relationship the operator is to have with this technology.
This has been a common trend in the subsea robotics industry since forever. “Capability” has been synonymous with hardware. We are all familiar with a capability statement backed up by an operator console with monitors everywhere, each running a separate software program controlling a separate sensor or payload. Truthfully, one needs to look no further than our own personal cars and cell phones. Cell phones provide an amazing navigation capability but are so distracting to use by the already task-saturated driver that their use during driving is illegal in most places.
If the operator is task-saturated, several things are going to happen. Training will achieve a basic level of proficiency for some but not all and it will be time-limited as fatigue sets in. Once the operator gets fatigued, performance degrades and safety becomes an issue. Training also becomes significantly more complicated because more time is required to achieve an appropriate level of aptitude. Task-saturation also limits the operator’s ability to utilize the product to its fullest potential. This can be catastrophic to the viability of the product, especially if the real differentiating features are those difficult to realize because the operator is too taxed. This can also be catastrophic to the operator if she is too task-saturated to effectively use the safety features of the device.
Through our introduction of RNAV2 into combat swimming operations, we saw first hand that modular capability is not always the right thing. We needed to increase capability with fewer modules and fewer things for the operator to do. Effectively, we understood that the only way RNAV2 would meet its potential was if the operator could not do something else. We had to address the entire system. We had to add capability but reduce the operator task load.
Believe it or not, the answer was to simplify it through the addition, not subtraction, of even more technology. The net desired result was to remove something from the diver’s workload. We could not remove anything from the RNAV2 interface, it was already sleek, efficient, and minimal. But, we could offload some other tasks from the diver. We could remove piloting the DPD.
Subsequent to the initial RNAV2 prototype rollout and less-than-awesome reception by the combat swimmers, we added autopilots, mission planning, and autonomy to the RNAV2 so that it could pilot the DPD and the operator could use the RNAV2. Regarded almost as a party-trick or high-dollar feature in the early days of RNAV2 planning, autonomy turned out to be the lifesaving (maybe literally) feature of RNAV2. With the autonomy system, pilots can simply select a programmed mission profile, program one on the fly, or use the fly-by-wire interface through the RNAV2 and let the RNAV2 pilot the DPD, freeing the diver up to focus on navigation, sonar operation, obstacle avoidance, health and safety factors, and doing the job at hand.
The next evaluation of RNAV2 by combat swimmers went entirely different than the first. Yes, we had to do more training because we had to show swimmers how to not only use RNAV2 but let RNAV2 fly the DPD so they could really use the RNAV2. (There were also some trust issues we had to overcome.) But, the end result was exceptional. We found operators could effectively use all of the features of RNAV2, training created a clear and consistent path to aptitude across all operators, and any operator could achieve the same level of required proficiency with the same amount of training.
Most importantly, the operators like RNAV2 now. They believe in it and they believe that it can help them do their jobs better. Anyone who has ever integrated a new product into a user group knows that the subjective like is critical to the success of the product.
“Completeness” in product development is sometimes a bit of an asymptotic process and we are always trying to improve RNAV2. Improvements mean adding capability and at the same time taking something away from the operator. We recently improved the usability of RNAV2, again, by adding a new navigation mode affectionately known as the “grip it and rip it” mode. RNAV2 has always provided accurate navigation through the use of a full inertial navigation system and fiber optic gyro but there were a few steps required to initialize and align the unit properly, especially if there was no GPS available. This initialization and alignment process has always been the hardest thing to learn. The 2021 software update introduces a new algorithm and user interface that allows the diver to skip the manual initialization and alignment tasks without compromising accuracy. I had the chance to dive RNAV2 last month and was blown away by how effortlessly I could achieve exceptional results. Another step forward because we listened to the operators.
Today, several years after the first RNAV2 evaluation by combat swimmers, STIDD and Greensea have RNAV2 units deployed in Special Operations Forces teams around the world. It is an exceptional product and well liked by operators. We spend a lot of time every year working with operators to understand how they work and what they need to improve their performance.
Through the development of RNAV2, our team learned a few key things that we apply to every project today. They have become important elements of the culture at Greensea.
- The sooner you can get a product in the hands of the actual operators, the faster that product will be ready.
- An 80% solution in the hands of the operator is better than a 100% solution in the hands of an engineer.
- We have two ears and one mouth for a reason. Listen carefully to what the operators are saying.