Object and Asset Mapping with AI - Jeff Robinson

Object and asset mapping with AI

Utility design, UX/UI

Role

Product Designer

Team

Product Team @ FogHorn

Duration

Jun-Dec 2021

Overview

Object mapping is essentially the process of associating or tagging the ports of physical devices with the underlying automation and control networks, no matter the physical layout of the site. This configuration map allows the system to monitor and control devices, to track and schedule energy consumption, to keep environments comfortable during occupied periods, and more.

This tagging process can be labor-intensive, especially at larger sites where there may be many hardware devices with many control points, and with complex scheduling needs across several buildings. This is why my team built our object mapping tool, including its AI-assisted auto-tagging feature.

ℹ️ FogHorn Systems was a leading developer of industrial and commercial applications optimized for industrial edge environments with limited or no network connectivity. FogHorn was acquired by Johnson Controls in January 2022.

Mission statement

To reduce complexity around the process of asset mapping, to enable users to see the relationship of their assets and objects at a glance, and to provide them with some AI-assisted tools to speed the mapping process overall.

Goals

To present the setup process in discrete steps, allowing users to manage their configuration in an intuitive way.

To present an inherently complex process in a clear and organized way
To make it easy to locate individual assets or objects within potentially massive datasets
To provide smart, AI-assisted functions to reduce monotony of an otherwise manual process

Architectural overview I illustrated

Who is our targeted user?

Site operators who need real-time insights about the health of their equipment, through dashboards and alerts.

Challenge overview

Asset mapping can be a cumbersome process inherently. The user must define not only the particulars of a physical site, but import a potentially large dataset of on-site devices and components for mapping. A lack of guidance in the process could lead to misconfiguration, overwhelming the user as they hunted for the cause, and increasing the work required to gain the promised insights.

Solution overview

By breaking the process down into highly-focused steps, each accessible at any point, users could be guided through each individual stage. Throughout the planning process and earliest design stages, we consulted users and key stakeholders to gain their insights, and to incorporate their needs into their product. The result was a streamlined flow that reflected user intent at every step.

My process

Since this was a new product with no prior versions, I acted as the product designer, working beside my seasoned, cross-functional team to provide the overall design vision based on abstract specs.

I crafted the overall look and feel, as well as the progression concept in the form of a wizard. I drew upon best practices for the mapping interaction model, using an object + inspector approach that would be intuitive for our expert users, and familiar. From wireframes to prototypes, and through to completed mockups and asset delivery, my responsibilities encompassed the entirety of the visual experience.

Object actions conceptual exploration

Conceptual exploration

Determining what the interaction model should be starts with the underlying concepts, from the data model to the technical requirements. Thinking procedurally and iterating over versions allows me to work through core questions with my team: how should the steps be sequenced, how should we present the information in an extensible way, and what controls should be surfaced where?

Early wireframe concepts

A little bit of this progression is shown above. The earlier sketch focused on the basic relationship of assets to objects, and what properties might be listed for each. The latter sketch reflects an evolution after many conversations my team had about how the user would actually go about the tagging of the assets and objects, from bulk selection of entities to the setting of their properties.

Site mapping conceptual exploration

The conceptual designs above show a similar exploration around site mapping, where the user would be defining the characteristics of the physical space that housed the equipment to be managed. Stepping through these conceptual models with my team proved useful for sussing out exactly what we needed to expose to users.

Early concepts: UI for site hierarchy and selection

User research

Throughout the project, we remained engaged with internal and external stakeholders to incorporate qualitative feedback into every stage of the implementation, and to ensure the project’s success. Their invaluable guidance was key to ensuring that the workflow and the tools met their exacting technical expectations, mirroring the data model, while also making potentially massive datasets accessible.

Understanding user needs allowed me to spearhead the design of this novel product. I iterated on the conceptual model based on customer insights, and was able to test them in real time with the user’s feedback.

A new experience

The user would be establishing the layout of their site—its floorplan—in our tool, so in the meantime I was exploring concepts for hierarchical workflows. Once defined, the user would need to be able to select any of those areas for tagging, and the approach would have to scale well for sites with potentially hundreds of hierarchical nodes.

Site setup properties editor

Once we’d established our design approach to the site setup, we provided the user with the means to tell us the “where” and the “what” of their site, by entering the properties of their floorplan, and uploading a list of the assets installed there. Our system had to be able to accommodate sites of all sizes and complexities.

Mapping assets and objects

My design for bulk-tagging of multiple assets alleviated that time-consuming tagging process for our users. But a new AI-assisted approach could speed the process up even more. I worked directly with the engineering team to visualize, design, and implement a one-click tool that could assess great numbers of ingested objects, and instantly assign the best standard matches, which users could easily override.

UI for AI-assisted tagging of multiple assets

Impact

While our users worked in high-security environments—we didn’t get detailed quantified insights such as increased sales—their full-throated praise and eager adoption of our solution made up for that. Our attention to their needs fostered an ongoing trust in them that our team was responsive and nimble, and had their best interests in mind.

Lessons

Throughout this experience, I was mindful to assess the key focus of each stage of the process—its inherent scope and limits—to prevent any risk of the user feeling overwhelmed. Empathizing with the user’s needs, and keeping their expectations at the forefront, was a crucial factor in presenting a highly technical utility in a way that feels frictionless.