Mobile app design for machinery training
Problem: The maintenance of the machinery is difficult to explain and to understand.
The maintenance of machinery in the medical waste processing plant is a highly complex process that requires years of specialized technical experience.
This circumstance makes it difficult to transfer knowledge from senior mechanics to more junior profiles.
With the aim of streamlining this training process and ensuring operational continuity, PreZero reached out to design and develop an app that would enable more efficient and effective training of its junior staff.
Research
After conducting a comprehensive evaluation of the plant’s situation and gathering feedback from employees and bussines department, we began by understanding the users’ challenges.
Our analysis of supervisors’ input revealed that users often struggled to remember the necessary steps for machinery maintenance. Additionally, for reasons of convenience or speed, employees frequently disregarded the mandatory safety equipment.
On the other hand, the operators expressed concerns about the lack of hands-on practice in the workplace.
The solution should be capable of monitoring staff, reinforcing workplace safety protocols, and, to a lesser extent, supporting training efforts.
With this first approach to the situation, we began by investigating the work limitations the plant might have, with the ultimate goal of presenting the client with a future MVP.
Given the staff limitations in Mense, a wireframe was created and presented to both the business team and the development team, seeking both technical and economic evaluation.
Once approved by the client and the different stakeholders, a high-fidelity mockup was created for its subsequent development.
App Home
The application was developed for local use within the plant, whether on the machines themselves or in training rooms. The plant already had maintenance processes focused on safety, so much of the development process centered around digitizing and making those processes traceable.
Homepage was designed to serve as a hub where users can easily access information and tutorials, in addition to the training modules themselves.
Positioning within the plant
The positioning of digital models directly over the machines was accomplished using anchors, which must align with images on the floor.
Initially, we considered using QR codes to position the models in space. However, this approach was dismissed due to the requirement of placing the models in different locations for each session.
Instead, we chose a SLAM based system (the foundation of ARKit and ARCore), which allows us to position the models anywhere, whether on-site or in the office.
Video player and document viewer
Training content modules was directly extracted from the manuals and supplemented with multimedia elements, such as supporting videos, to enhance user understanding.
Regardless of the machine on which the training is conducted, the interface remains consistent, promoting a quick learning curve and allowing users to focus on the content.
Steps to follow to achieve the goal
Throughout the training, safety measures are emphasized, often highlighted with specific steps such as requiring participants to wear gloves or ensuring that machinery is turned off and that no one is nearby.
The application clearly indicates operational zones using an orange glow.
In situations where training occurs directly on the machinery, the digital model is hidden, with glowing indicators on the actual machine marking the operational areas. This approach makes it much clearer where work needs to be performed.
Additionally, it enables users to overlay the required steps directly onto the physical machine. By being connected to a cloud-based system, it gathers data, enhances performance, and streamlines machine management.
POC Hololens
During the project, the feasibility of using HoloLens was studied, as it would allow operators to have their hands free.
To validate the potential use of this device and identify any drawbacks within the plant, a Proof of Concept (POC) was created.
The application was ported to HoloLens, and the menus were adapted for use on the device.
Main menu
The HoloLens augmented reality glasses are operated using hand gestures, but they do not detect very good collisions between hands and digital objects. This limitation presented a challenge since the system we had developed was primarily designed for use with tablets.
To address this, we decided to redesign the click and selection system to enable full functionality without the need for controllers.
Our solution involved implementing time-based buttons. In this approach, the user hovers the cursor over a button, and an on-screen indicator fills up to confirm and execute the action.
As an assistant, the main screen follows the user and can be deployed when needed, allowing it to be moved out of the worker’s field of vision to avoid distraction during tasks.
Login and selection of training
The login was used as a means to identify which user was undergoing the training.
Another challenge we encountered was user inputting text. To mitigate this friction, we leveraged the device’s built-in voice recognition and transcription functionality.
This function was made optional for sensitive content.
Since this was only a proof of concept, it was conducted with a single training session.
Guidance screens for training and placement of the digital model in the real space
The training guidance screens were designed similarly to the tablet version: simple, with digitized content and step-by-step instructions.
A strong emphasis was placed on safety measures, which are now more tangible due to the device.
In this context, the use of HoloLens headset versions was studied, as they were being launched into the market at that time. It was dismissed due to cost considerations.
Ultimately, hololens device was dismissed for several reasons, including the device’s limited adaptability to dusty environments and the complexity of wearing the glasses for extended periods during maintenance activities.
The result:
As a result, users can conduct training sessions in the learning rooms, leveraging the Digital Twin technology for simulations and real-time feedback.
.
Maintenance times were reduced by 20%, as new operators can perform maintenance tasks at the same level of efficiency as the more experienced ones, thanks to the Digital Twin's predictive capabilities.
.
The supervision of operators and machinery is now much easier and more efficient due to the integration of Digital Twin models for enhanced monitoring and diagnostics.
.
The project included an application addressing similar topics, but it approached them through the use of virtual reality These solutions were seamlessly integrated into the existing platform.
.
In this project, I learned the importance of tailoring solutions to the specific environment in which they will be implemented. Additionally, I discovered that certain technologies, even if initially designed for professional settings, cannot be deemed effective until they are thoroughly tested in real-world scenarios.
Main technics in this project:
Prototyping and Wireframing, Interaction design,Accessibility (WCAG), User research and analysis (Google Analytics, Hotjar, Test A/B, Interviews...), Responsive design, AGILE, SCRUM and design thinking, User-centered design (UCD),Design systems (Libraries, tokens, documentation...), User research techniques and analysis (Google Analytics, Hotjar, Test A/B, Interviews...), AGILE, SCRUM and design thinking and User-centered design (UCD)
Are you interested in new tech projects, check this project: