The desktop shell

The desktop shell is the Studio in a real Windows window. It renders the same browser page from the same server. It adds the one gesture a web page cannot offer, because a browser cannot see the desktop. You rest the cursor on a control in any application. The shell writes down what is under it as a locator.

#What the shell is

The shell lives in studio/desktop-shell and builds to TaskSultanStudio.exe. It is a .NET 8 WPF application with a WebView2 control filling the window. It points at one origin, http://localhost:4174 by default. It lets the Studio server serve both the built page and the API there. Nothing about the Studio is re-implemented in the shell: no view, no route and no state of its own. A second set of views would be a second thing to keep in step. That is what this approach avoids.

The one thing the shell does add is the recorder's own UI Automation engine. It links Uia.cs and Native.cs from the recorder project rather than copying them, so the element it picks and the chain it builds come from the same code a recording uses.

#The indicate gesture

There are two ways to trigger the pick: the "Indicate element" button in the strip at the top, or the Ctrl+Alt+I hotkey registered by the window. Both do the same thing. The shell reads the current cursor position, asks the accessibility tree what element is under it, then walks up from that element to its top-level window, building the chain of levels in between.

Nothing is clicked, typed or moved. The hotkey is a registration, not a key press sent to any application. The pick only reads. Indicating a button cannot press it, which is a product rule rather than a preference: synthetic input is not allowed. The result of a pick is a locator rather than a demonstration.

#How the locator comes back

The pick produces a JSON document and writes it to two places: the path named by --out (default %TEMP%\tasksultan-last-locator.json) and the clipboard. The document carries the source, the time, the cursor position, the control's identity, its window and the full chain.

The field you use is locator. It is written in the shape a target file uses:

{
  "name": "Eight",
  "controlType": "Button",
  "automationId": "num8Button",
  "within": { "name": "Calculator", "controlType": "Window" }
}

Leaving automationId empty is honest: when a control has no id, the chain is the address, which is what a recorded step carries. The shell does not post this locator into the running Studio itself. It writes the file and sets the clipboard. You paste it into the Studio, a target or an automation. The Studio's own Desktop Recordings tab has a separate Indicate button, which drives a dwell indicator through the API instead.

#The strip, reload and what is unfinished

The strip at the top names the URL being shown and reports what the shell is doing. It is also the shell's only reliable instrument: WebView2 is GPU-composited. A normal window capture renders it as background, so a screenshot is not evidence for this window and the status line is. The Reload button, F5 and Ctrl+R all reload the page.

The launch arguments are --url, --log and --out, plus --pick-now, which runs the pick once as the window opens so the mechanism can be checked without a hand on the mouse.

This part of the product is early. It has a handful of commits, the pick writes a file and the clipboard rather than talking to the Studio server. There is no locator library inside the shell. Treat it as a working hand-off, not a finished recorder.

#Things that catch people out

• If the page comes up blank, the web control did not start. A previous shell's WebView2 processes can still hold its user-data folder in the temp directory; clear them and relaunch.
• The strip reports the load result. If the Studio server is not running, the strip says so rather than failing.
• Run the shell against the port the Studio server is actually on. The default http://localhost:4174 matches npm start.