Main Balder components¶
The following section should help getting an overview over the available components in Balder. You will learn the key facts about Scenarios and Setups and how their Devices work. You will learn what Features are and how Balder matches these between Scenarios and Setups. You will also learn how to connect devices with each other over Connections and how Connection trees are defined.
Note that this section only provides an overview of the components. You can find a detailed description of each element in the Basic Guide.
Difference setup and scenario¶
The basis of Balder is based on individual scenarios and setups that are in a fixed relationship to each other. Scenarios define a situation under which individual tests are carried out. Setups, on the other hand, describe how a test environment currently looks like. Two of the most important statements in Balder are:
Scenario: Describes what you need
Setup: Describes what you have
So what does this mean? Take a closer look into these definitions.
Scenario: Describes what you need¶
A Scenario always defines what you need. So for example if you want to test an online login on a server, you need the server and a client device that tries to connect to the server device. These are the components of a Scenario. It describes what you need. In this case all other devices that are connected with the network don’t matter.
Setup: Describes what you have¶
It is different when we look at the Setups. In a setup you define everything that is available and relevant in the environment, the particular setup is for. So for example, if you have your computer, the router and the server of company X in your influenceable spectrum of devices you can add all of them to your setup. Also if the scenario is written later only for the router and the server, it will work out, because Balder will automatically match scenario-devices with compatible setup-devices.
What are devices?¶
A device can be a component of a Setup or of a Scenario. In generally it describes a container object, which represents a test component, an application or a real physically device. Generally, a device can be everything which has functionality.
In Balder, devices are inner-classes of Setups or Scenarios. These classes have class properties that
describe instantiated feature classes (later more). This easily allows you to use these features
by referencing the Setup or Scenario devices in your test. You can access the
properties with self.MyDevice.*
.
Note
Note that a device itself never implements something by itself! A device should only have class-attributes which hold an instantiated Feature object. More about this later on.
Functionality = Feature¶
A device is a collection of so called Features. Every feature stands for a functionality the device has. So for
example a BrowserDevice can have OpenAWebpageFeature()
, which describes the functionality to interact with a
website. It does not provide the site itself. The website should be provided over another ServerDevice
which uses something like a ProvideWebpageFeature()
.
In Balder you have to define scenario-devices similar to the following example code:
class ScenarioMyOwn(balder.Scenario):
...
class BrowserDevice(balder.Device):
webpage = OpenAWebpageFeature()
class ServerDevice(balder.Device):
provider = ProvideWebpageFeature()
...
# you could use the device in a testcase
def test_webpage(self):
addr = self.ServerDevice.provider.get_address()
...
Note
Please note, that Scenario
classes must be defined inside files that start with scenario_*.py. In
addition their class name has to start with Scenario*. Otherwise the file will not be picked up by Balder.
How to connect devices¶
In the real world, devices are connected with each other. If you have a BrowserDevice and a ServerDevice like mentioned before, you could expect that these are connected with each other over something like a HTTP connection. For this, Balder provides Connections.
Simple connections¶
Balder is shipped with a lot of different connections (see Connections API). In addition, you can create your
own ones, by simply inheriting from the master class Connection
.
from balder import Connection
class MyOwnConnection(Connection):
pass
Connection trees¶
The connection-tree is a global hierarchical structure, that describes how connections are arranged with each other. For
example that a HttpConnection
is based on a TcpConnection
which itself is based on IpV4Connection
or
IpV6Connection
.
The whole thing allows you to define subtrees, that can be used to connect devices. You can read more about this in the section Connections.
Connections between devices¶
We have learned a lot about connections and how they are organized, but how do we connect some devices with each other?
This is really simple, because Balder provides a decorator connect(..)
here. If you want to connect the two devices
BrowserDevice
and ServerDevice
with each other, you can do the following:
import balder
class ScenarioMyOwn(balder.Scenario):
...
class BrowserDevice(balder.Device):
...
@balder.connect(BrowserDevice, over_connection=HttpConnection.based_on(IpV4Connection))
class ServerDevice(balder.Device):
...
It works the same way in setups.
How setups work?¶
So far we have defined the so-called scenario level (what we need). But of course we also have to define the actual
real environment that we have. For this we use the Setup
classes.
As mentioned earlier, Setups always describe what you have! Similar to Scenarios you define all your devices and add features to it. But here you can define everything you have or you want to use in the testenvironment. Balder will automatically determine (based on the feature set and the connections between the devices) in which constellation a scenario will fit to a setup.
Implement features¶
Often scenario-features don’t provide the whole implementation. In most cases, these features are abstract and a user specific implementation has to be provided on setup level (means, in a child feature that is instantiated inside a setup devices). For this, you can add a new module features_setup.py into the setups directory, that provides the specific setup level features:
|- tests/
|- scenarios/
|- ...
|- setups/
|- features_setup.py
|- setup_my.py
|- ...
Note
Balder is not interested in where you implement the feature objects. Feel free to use your own structure, but be careful, where you save your files you want to reuse. Without a clear structure this can be a little bit confusing.
Note
If you have many features you can also use a python module. For this you create a directory with the name
features_setup
and add a __init__.py
file in it.
|- tests/
|- scenarios/
|- ...
|- setups/
|- features_setup/
|- __init__.py
|- my_example_feature.py
|- ..
|- setup_my.py
|- ...
Here you can implement many files in it, which allows you to separate the features a little bit.
How does Balder know, which feature you are implementing?¶
Maybe you ask yourself how Balder knows which scenario-feature you are implementing in your setup. For this Balder uses Inheritance!
The scenario-features often implement abstract properties or methods. You can implement them easily by overwriting them.
The file features_setup.py
for example could have the following content:
# file tests/setups/features_setups.py
# import from global lib - has the abstract feature that is directly used in scenario-device
from ..lib import MyAbstractExampleFeature
class MyExampleFeature(MyAbstractExampleFeature):
def do_it(self):
print("I do it")
The setup file itself, defines all the devices you have in your testsystem and adds the features to it in the same way like it is done in the scenario, but of course with the subclass, that really provides the implementation:
# file tests/setups/setup_my.py
# import from setup feature file
import balder
from .features_setup import MyExampleFeature
class SetupMy(balder.Setup):
# also inherits directly from `balder.Device`
class DeviceDoer(balder.Device):
example = MyExampleFeature()
...
class OtherDevice(balder.Device):
...
...
Note
Please note, that Setup
classes must be defined inside files that start with setup_*.py. In
addition their class name has to start with Setup*. Otherwise the file will not be picked up by Balder.
You can implement more devices than in the scenario, Balder doesn’t care. It will search for devices that match the requirement, defined in scenario. If the matching candidates have a matching connection-tree and if all required features of a scenario-device are also implemented in the setup-device, Balder will run the scenario-testcases with this constellation!
Note
Note that test methods have to be defined in scenario classes only. Setups don’t support own test methods!
How does this work together?¶
It is really important to know the difference, so we want to repeat it again. The most important differences between scenarios and setups are:
Scenario: Describes what you need
Setup: Describes what you have
These are the golden rules, Balder works with. After you have defined a scenario, add some devices to it and instantiate their feature objects as their class attributes. This describes what your testcase needs.
Then you think about what you have. How does your test rack or your test pc/pipeline look like? All this can be defined in a setup. Add every device you have and implement your features for them. In the same way you have defined the scenarios, you have to instantiate your implemented features in the setup devices.
Matching process¶
When Balder is executed and after it has collected all relevant classes, the matching process takes place. It determines which device-mappings (between scenarios and setups) match with each other. For that, Balder is interested in the feature sets your devices have. Based on these feature sets, Balder will automatically determine the possible mappings between the Scenario-Devices and the Setup-Devices.
Generally, Balder will create matching candidates by searching possible mappings of a scenario device with one of the available setup devices. In this stage Balder does not care if the devices are compatible.
Feature check¶
The first filter stage will remove all mapping candidates where one or more setup devices don’t provide all features the related scenario device has.
For example we have the following inheritance structure:
balder.Feature -> MyAbstractFeature1 -> MyFeature1
balder.Feature -> MyAbstractFeature2 -> MyFeature2
balder.Feature -> MyAbstractFeature3 -> MyFeature3
Now we have the follow matching candidates
ScenarioDevice: <=> SetupDevice:
MyAbstractFeature1() MyFeature1()
MyAbstractFeature2() MyFeature2()
This would work because the MyFeature1
is a subclass of MyAbstractFeature1
and the MyFeature2
is
a subclass of MyAbstractFeature2
.
The following example would also work, because the same features are allowed as well
ScenarioDevice: <=> SetupDevice:
MyFeature1() MyFeature1()
MyAbstractFeature2() MyFeature2()
Now let’s also take a look at an example that does not work:
ScenarioDevice: <=> SetupDevice:
MyAbstractFeature1() MyFeature1()
MyAbstractFeature2() MyFeature3()
This would not work because there is no feature in the setup, that implements the MyAbstractFeature2()
!
What about having more features in the SetupDevice
than in the ScenarioDevice
?
ScenarioDevice: <=> SetupDevice:
MyAbstractFeature1() MyFeature1()
MyFeature2()
MyAbstractFeature3() MyFeature3()
This would work, because we have an implementation for MyAbstractFeature1
and for MyAbstractFeature3
in the
SetupDevice
. It is not important that the scenario device doesn’t provide a parent class for the MyFeature2
of
the SetupDevice
. For this current mapping, we are not interested in it. Remember, the scenario describes
what we need, the setup describes what we have. If we have more features implemented in our setup device, it is
ok and we are not interested in this for the current mapping. Maybe we will need it in another mapping with another
scenario later.
Connection Sub-Tree Check¶
With the feature filter we have already filtered a lot of candidates. Now we are interested how the devices are connected with each other. For this, we check if the connection-tree, that has been defined in the scenario, is contained in the connection tree, that has been defined in the setup. This will be done for every connection between the matching devices. Every matching with one or more device-connection that does not pass this, will be filtered.
Execution¶
In the last step Balder will execute the mappings. You can execute Balder, by simply calling it inside the project directory:
$ balder
+----------------------------------------------------------------------------------------------------------------------+
| BALDER Testsystem |
| python version 3.9.5 (default, Nov 23 2021, 15:27:38) [GCC 9.3.0] | balder version 0.1.0b5 |
+----------------------------------------------------------------------------------------------------------------------+
Collect 1 Setups and 1 Scenarios
resolve them to 1 mapping candidates
================================================== START TESTSESSION ===================================================
SETUP SetupMy
SCENARIO ScenarioMyOwn
VARIATION ScenarioMyOwn.ClientDevice:SetupMy.Client | ScenarioMyOwn.ServerDevice:SetupMy.Server
TEST ScenarioMyOwn.test_webpage [.]
================================================== FINISH TESTSESSION ==================================================
TOTAL NOT_RUN: 0 | TOTAL FAILURE: 0 | TOTAL ERROR: 0 | TOTAL SUCCESS: 1 | TOTAL SKIP: 0 | TOTAL COVERED_BY: 0
Balder automatically detects valid variations between every scenario and the existing setups. For every mapping all tests of the scenario will be executed.