Connecting the Physical with the Digital in RPA implementations

Where Robotics and Machine Learning Meet
19/11/2018
VASS moves to the forefront of Software Robotics In Europe and Latin America partnering with Jidoka
15/01/2019
Show all

Connecting the Physical with the Digital in RPA implementations

As we are moving into an era where Digital workforce of Software Robots is starting to become an essential part of the workplace across businesses and institutions, we at Jidoka are re-imagining how to create synergies between the human and the digital workforce by creating innovative solutions combining RPA with other major 4th industrial revolution technologies such as Internet of Things (IoT), Conversational Artificial Intelligence (AI) voice assistants, and Mobile computing.

Creating Ambient Intelligence at Workplace

In computing, Ambient Intelligence is a concept, that has been around for 20 years, which is about creating an environment using technology and devices that are sensitive and responsive to the presence of people. The Technology devices embedded in the environment provide the machine intelligence that supports the people in carrying out their day to day activities in a way which is very natural to humans, by utilizing the information and computing power which is hidden in the network behind those devices.

We’re now embarking in a technology driven world where on one hand the use of physical devices empowered by technologies such as IoT, AI and mobile is becoming more and more pronounced in our day to day life. And, on the other hand, Intelligent Automation and Software Robotics is emerging as the newly found machine intelligence at workplace. Converging the two, the power of the physical devices and that of the digital workforce, it is about bringing to life this whole idea of Ambient Intelligence in the Workplace environment.

An important concept in Ambient Intelligence is Ubiquitous Computing i.e. Computing that is made to appear anytime and everywhere. In the context of RPA and Intelligent Automation, it is about creating a “Ubiquitous Digital Workforce” that is accessible anytime and everywhere within the workplace environment. This can be made possible using technology devices, that allow multimodal interactions, i.e. interactions that happens through natural modes of communication such as touch, voice or context-awareness, between the digital and the physical environment. Multimodal interactions, in making the Digital Workforce ubiquitous, are possible by using devices that are empowered with technologies such as IoT, Conversational AI or mobile computing technologies.

We at Jidoka are building technology solutions to make this happen using physical devices that are available at very affordable commercial prices. For us, creating Ambient Intelligent Automation Solutions is all about creating synergies between the human and digital workforce. And this is important as Digital workforce starts to scale across businesses and organizations, to build a perfect bonding between the two, for a fuller employee engagement as we move into a world of RPA and Intelligent Automation, and that these automation technologies get utilized in the most effective way.

Jidoka Cobots – Creating Human-Robot Collaboration


At Jidoka, we are heavily focused on innovation and in recent times we’ve built automation solutions around this theme of human-robot collaboration under the category of Collaborative Robots or “Cobots”. In the article post, we would like to share details about three of such technology integrations with our RPA platform.

IoT Integration using Amazon IoT device- RPA Panic Button

The first solution is about integration with IoT technology, in this case by using an Amazon IoT button device which is a cloud programmable dash button, that allows with a button click a pre-configured action to happen. Amazon IoT button can have many usage examples such as to start or stop a service or application, to call someone or generate an alert, etc.

Within the context of Robotic operating environment, we have configured this device into our platform to be used as a Panic button or a Red button which can be used to disable or re-enable all robot executions from the RPA platform in case there is an emergency just with a click of a button, from any location, any network. Given we have a heavy emphasis on event-based robot executions and exception handling in the Jidoka platform, we enable configuring specific global actions to be triggered through the Amazon IoT button click events. When an administrator operates the panic button, the information from the IoT device gets sent to the AWS IoT cloud service through the internet. And from there, it gets re-routed as secure messaging to the specific Jidoka console server instance to which the device has been configured. A real-life use of the Panic button is presented in the following short video:

Conversational AI Integration using Google Voice Assistant – Hands Free Robot Operations

The second one is about integrating with Conversational AI technology, in this case by using a Google Voice Assistant device, that allows to invoke a voice agent that we’ve built at Jidoka called Doka.

Once the Google voice assistant device has been configured on a network, using the Google software, we then connect the device to a Jidoka console instance by activating a network link that allows information to be sent over from the device to the Jidoka server over a Wi-Fi or a mobile network.

Google assistant device starts to act as an intermediary between Doka assistant and human workforce when invoked using command “OK Google, talk with Jidoka Assistant”. As the user then talks to the Google device, Doka assistant applies Natural Language Processing to understand user’s query and in response it executes actions or commands on the Jidoka console and returns information back to the user. Internally it makes REST API calls to the Jidoka console. And since we provide an extensive REST API for our RPA platform, there are a variety of actions that a user can achieve by talking to the Doka Assistant for a hands-free management of robotic operating environment at the workplace. Some examples can be checking the console status, navigating through different modules, checking what nodes are available to execute robots, searching and executing specific robots. For an easier understanding, please watch the following short video how Doka can be brought into action using Google Assistant device:

Mobile Integration using Jidokaphone App – Execute Robot and Manage Exceptions on the Go

The third solution is about integrating with Mobile technology, in this case by using our Mobile app called Jidokaphone.

A user can either use the Jidokaphone app to execute commands on the Jidoka platform through voice, or can interact with the platform via a chat interface by typing text, or by using the native mobile app interface to perform specific actions via touch screen. A typical use of Jidokaphone app is to allow users to easily search and execute a specific robot with inputs that can be in the form of a file or text information, and once the execution has completed to see the results of the robot execution on the mobile app itself along with any output files that are generated.

Another interesting feature is to allow for interactive exception handling during a robot execution, for example a robot asking for a human to resolve a captcha phrase, as shown in this picture. In the Jidoka console individual robots can be configured to send information to the mobile app of a user when certain events occur during the execution. The mobile app is connected to an instance of the Jidoka console typically with the server hosted in a cloud environment. As such the mobile app can be used to execute robots on the go, anytime, and from anywhere as shown in the short video below.

Deepak Sharma
Deepak Sharma
RPA Consultant, Specialist & Evangelist

Leave a Reply

Your email address will not be published. Required fields are marked *