QA Guild emerges from shadows!

Our team deals with incredible projects, exciting challenges and great people. But we also provide enormous support for each other. Here, one can receive help, but most importantly: offer their help to others. This is why we have created the Guild. It is a community that collects and figures out interesting, surprising, and troublesome situations. A community that shares knowledge about new and ingenious tools or solutions. We all know it: two heads are better than one. And a whole team of head… is more than two. 

First, the Guild will step out of the shadow. Let us talk about Shadow DOM!

What has happened?

Last month, for one of our clients, we prepared tests for an application written in a solution named Vaadin. Below you can see an example of a similar application:

In our exercise, we will try to find <input> element using a search engine on the left side with Serenity BDD framework that uses Selenium WebDriver. But first, let us start with browser development tools. There we should find this:

We do not see here anything that would look suspicious. Let us look above:

When we search for <vaadin-text-field> it turns out that it does have <input> inside.
But when we enter the xpath //vaadin-text-field//input selector and confirm – nothing happens. The element is not found.

Why did that happen?

Why is my test not finding the analogous element? I was wondering about it for a long time! Especially that in tests for the client we were using type id selectors, so it should be even easier.

And actually, the reason is very simple (of course: once found :P) It is the “shadow DOM”, in which our element is located.

It is a similar mechanism to the one, which worked when you were embedding one HTLM page in another. In that case, an iframe was used. The downside was that sometimes the CSS could interfere between the parent and the iframe. It was unlikely, but it happened.

We can assume, using this analogy, that shadow DOM is a more elegant iframe. It makes sure there will be no interferences between the external and internal world. It is used to encapsulate components in the DOM structure.

You can find more professional explanation in the documentation. In a related article we will find an interesting diagram illustrating the whole mechanism:

What is the problem?

We have to find a way, to get to the chosen element in shadow-root with Selenium. How? Unfortunately, there are no built-in “helpers” for such problems. The only working solution I have found is to add a JavaScript snippet in the test. First, we find the <vaadin-text-field>, then we “expand” the shadow root. 

When we come back to the elements, we see that shadow-root is attached to <vaadin-text-field>, which in this structure is a host.

Proposed solution

Let’s check Javascript, which finds the <vaadin-text-field> element and “expands” the shadow-root element connected to it:

This snippet returns us an element that contains everything we did not have access to before, as seen in the screenshot above.

Simple? Seemingly simple, but still a bit problematic. After all, from the WebDriver’s point of view, we should not really touch the script. But it seems we have no choice.

A few methods

ExpandRootElement Method

It somewhat works but makes testing more difficult. What is more, this script will not work correctly in geckodriver (a driver that supports Firefox). The bug was already reported on GitHub.

We can work around it, calling for the children method:

This works, and the results look like that (we will see that in developer tools):

However, it is easier to search elements in a single element-parent, than in a collection. That is why I wrote one more snippet, which lets you go through the whole collection, throw away tags that are styles, and in next attributes download innerHTML and so on. As you can see, it got very complicated.

As it turned out, in the Vaadin application even finding the element does not mean we were successful.

In the construction used by Vaadin, even if we find a shadow-root, input and so on, we will not be able to successfully enter the text using type method from Serenity BDD (it is a wrapper of sendKeys method from Selenium WebDriver). We can “type” the text with the set value. I do not like that solution, but sometimes you have to do things you do not like. However, if the inputs have validations attached, the text – even though it was entered in the input (visible in the browser) – may not be validated positively.

Why is it not validated? In TextBench, a Vaadin’s creators testing framework, I checked an implementation of their solution. First, authors set a value for the element, and then call for Java Script events below it, to stimulate the user’s action.

In the beginning, I was wondering why Vaadin’s creators write a paid tool for browser tests that is based on Selenium WebDriver. Why rediscover the wheel? It turned out that they did it in a different way. They took all that was possible from Selenium WebDriver, and then added a lot of JavaScript snippets that let you work around the difficulties of Vaadin’s nature.

As you can see, in the case of Vaadin, we are forced to test with dedicated tools. Otherwise, we will have no way to simulate interactions. And if we do this, such a simulation will not be a faithful representation of the actions performed by the user, because we use fragments of JavaScript code executed directly in the browser.

It was the first time I encountered such a problem when I was preparing a Proof of Concept for the client. This is why I am alerting you to the possibility of shadow-roots appearance. I hope that thanks to this material, in a similar situation you will be able to deal with it faster and – without any problems – get the elements out of the shadow, that is… Shadow DOM.