Mouse presence sensing web page

Hi.
Have a particular issue that a WF works beautifully when executed from PS but when executed from the agent through RDP it does not and have “detected” the reason visually but do not know how to solve this and hope to get some insights through the community.

WEB page being managed has a dynamic virtual keyboard and when a key needs to be pressed, it obscures all the characters with “*” when the mouse covers the keyboard.
I managed to create a Javascript Injection that does the typing and Surface detect to “move” the “virtual” mouse over the keyboard to activate the obscuring process. Typing does not work when characters are present.

When executing through the agent with RDP, it does everything beyond the typing on this keyboard and will only work when the mouse image is present on the screen,
Let me explain. On dual monitor, on with RDP on the agent station and my computer screen. When mouse is on computer screen, it does not type, but as soon as the mouse moves over to the RDP screen, it starts typing.

Before trying something else, like using surface plugins to type each key through image processing, would like to find a programmatic way of doing so.

Question, does AutoIt does a better job to simulate presence of mouse on a RDP environment or will it be the same as Surface. Any other plugin?

Any suggestions are welcome.

At the end it was a timing issue.
As I was gathering data from the web page through JSI, this data was not present at the time the data was being gathered so the following WF did not know where the keys were located on the keyboard. Example below of the numeric keyboard. Alphanumeric has same behavior of not being standard layout and keys are obscured when mouse moves over them.
Teclado Numerico

2 Likes