Advanced scripting tips and tricks in Reveal

This document adds information for programmers interested in:

  • developing scripts interactively in the python console and wanting to learn about debugging issues effectively,
  • developing scripts that interact with the GUI part of Reveal Chromatography, and draw information from it or modify it.

Debugging tips and tricks in the Python console

Turning on logging

As you explore your way through a custom analysis, there might be important messages and warnings that the Reveal Chromatography code will try and issue. One possible sign of this is the Python console issuing the following message:

No handlers could be found for logger "kromatography.<snip>"

You can suppress these messages, and turn on Reveal’s ability to log warnings and errors by initializing logging (and other useful tools):

>>> from kromatography.utils.api import initialize_reveal
>>> initialize_reveal()
2017-03-19 11:08:14 WARNING  [kromatography.utils.api:23] REVEAL Chromatography tools initialized.

Afterwards, rerunning the same code will display any warning and error messages and help with debugging issues.

Logging may even be done to see not just “warning” and “error” level messages from Reveal, but also “info” and “debug” level messages. That can help track what is happening in finer details, and maybe help understand an error:

>>> initialize_reveal(verbose=True)

Once that is not longer needed, one can re-issue a normal call to initialize_reveal() to reduce verbosity, which is equivalent to:

>>> initialize_reveal(verbose=False)

Debugging an Exception

At other times, users and developers’ command may lead to an exception to be raised. That is the sign of an error in the Reveal code which the application cannot recover from, and must lead to the failure of that command. That is a very normal part of using an API, and there are three things to know to not let that issue prevent from building the tool needed:

  1. An exception in Python isn’t something that will crash it and lead to loss of work: it is almost impossible to crash Python, and an exception just means that the command leading to it wasn’t used as expected.
  2. The exception is often by two sources of information that are very valuable: an error message that might explain what went wrong, and a Traceback, which shows the succession of calls that led to the exception. For complex commands doing a lot of things, that might help narrow down what part of the command failed.
  3. The Python console has a magic command which can be invoked after an exception as been raised, which will take the user back to right before the exception is raised using a command line debugger. A new debugging prompt then should appear allowing users to print variable values or move around in the stack frames. Please refer to the dedicated documentation to learn more about the pdb debugger. ipdb works the same way but supports tab-completion and nicer outputs.

For example, let’s say that one is trying to create a simulation from an experiment, but accidentally passes the name of the experiment instead of the experiment itself to build it:

>>> from kromatography.io.api import load_study_from_excel
>>> study = load_study_from_excel("Example_Gradient_Elution_Study.xlsx")
>>> print([exp.name for exp in study.experiments])
['Run_3', 'Run_2', 'Run_1']
>>> from kromatography.model.factories.api import build_simulation_from_experiment
>>> build_simulation_from_experiment?
Signature: build_simulation_from_experiment(experiment, binding_model=None, transport_model=None, name='', fstep='Load', lstep='Strip', initial_buffer=None, lazy_loading=False)
Docstring:
Build a Simulation object given an experiment and some models.
<snip>
>>> sim = build_simulation_from_experiment("Run_1")
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-11-53be2468d9ba> in <module>()
----> 1 build_simulation_from_experiment("Run_1")

/Users/jrocher/Projects/Kromatography/kromatography/model/factories/simulation.pyc in build_simulation_from_experiment(experiment, binding_model, transport_model, name, fstep, lstep, initial_buffer, lazy_loading)
     59
     60     if not name:
---> 61         name = generate_sim_name(experiment.name)
     62
     63     # NOTE: The number of components is : product components + cation

AttributeError: 'str' object has no attribute 'name'

From all this, one can see that the error was generated by the command:

>>> sim = build_simulation_from_experiment("Run_1")

that the error is of type AttributeError, and that that happened inside the kromatography/model/factories/simulation.pyc file and inside the build_simulation_from_experiment(). If one wants to jump at the moment right before the exception was raised, one can issue the following command:

>>> %debug
> /Users/jrocher/Projects/Kromatography/kromatography/model/factories/simulation.py(61)build_simulation_from_experiment()
     59
     60     if not name:
---> 61         name = generate_sim_name(experiment.name)
     62
     63     # NOTE: The number of components is : product components + cation

The arrow indicates that the code is about to execute name = generate_sim_name(experiment.name) and we already know that that will lead to the AttributeError. So it might be useful to look at the nature of the experiment object and that indeed, the experiment we passed doesn’t have a name since it is a string:

ipdb> p experiment
'Run_1'
ipdb> experiment.name
*** AttributeError: 'str' object has no attribute 'name'

The a command will list all arguments passed into the current function:

ipdb> a
experiment = Run_1
binding_model = None
transport_model = None
name =
fstep = Load
lstep = Strip
initial_buffer = None
lazy_loading = False

We can go up on the stack frame to see how that function was called, and who passed the wrong experiment argument:

ipdb> u
> <ipython-input-14-53be2468d9ba>(1)<module>()
----> 1 build_simulation_from_experiment(exp.name)

Since experiment was directly passed from the command we issued in the console, it might be time to abort the debugging to go back and look at the documentation of the build_simulation_from_experiment():

ipdb> q
<snip>

>>> build_simulation_from_experiment?

so that we can learn that this function expects an Experiment, not just a name.

Interacting with the application from a Reveal script

We introduced in section Reveal’s Python script runner the in-app scripting tool. The trivial “hello world” example there showed that the tools allows to run any standalone python commands. Consequently, any and all of the examples of using the Reveal code provided in the developer reference’s Using Reveal Chromatography’s API and any code that is developed interactively in Reveal’s Python console can be run (and/or saved) as a Reveal script.

This document focused on going beyond standalone commands, and accessing the currently running application’s elements, since that’s a functionality that is not available in the standalone Python console.

An example

To provide a more interesting example of how custom scripts can interact with the Reveal application, users can look at the content of the sample script included in the Reveal Chromatography called first_script:

""" Example script changing attributes of the application: study, user data,
...
"""
from kromatography.model.api import Product, Simulation

# Example of changing the content of the datasource by adding a new product
new_prod = Product(name="PROD002", product_type="Mab")
user_datasource.set_object_of_type("products", new_prod)

print("Current Study contains {} simulations".format(len(study.simulations)))

# Create a new simulation and add it to the study:
new_sim = Simulation(name="new")
study.simulations.append(new_sim)

# Open the new product in the central pane:
task.edit_object_in_central_pane(new_prod)

Let’s go over this script line by line to clarify what each line does. We remind the reader that any line starting with the # symbol in Python is a comment, and there to make the script more readable. First the block starting and ending with triple quotes just provides a description of the script and have no impact on execution: it is just there for user to track the purpose of this file:

""" Example script changing attributes of the application: study, user data,
...
"""

The next line imports the Product and Simulation classes since the script will create a new product and a new simulation:

from kromatography.model.api import Product, Simulation

The next 3 lines create a new and trivially simple product called PROD002, and adds it to the user data:

new_prod = Product(name="PROD002", product_type="Mab")
user_datasource.set_object_of_type("products", new_prod)

That is done by accessing the user_datasource object which controls the application’s user data (see below in How does script running works for more details). All scripts have access to it automatically.

The study data on the other hand is controlled by the study object, which any scripts has access to as well. One can, for example, ask to print the number of simulations the study currently has by printing the length of it’s simulations attribute:

print("Current Study contains {} simulations".format(len(study.simulations)))

But one can contribute new simulations to that list as illustrated on the next two lines:

new_sim = Simulation(name="new")
study.simulations.append(new_sim)

The last line of code in the script accesses a third object automatically provided to any script: the task, which can be thought of as what controls the main window. One of its abilities is to open (or close) tabs in the central pane. That is used to open the newly created product in the central pane:

task.edit_object_in_central_pane(new_prod)

This task object is a kromatography.ui.tasks.KromatographyTask, and its public interface can be explored in the documentation. Its purpose is to support a window of the main application (there is one task per window opened), and it controls of all its aspect: window title, menus and menu bars, central and dock panes, ...

If you run that script you should see the following happen:

  1. the script should run without errors, and print the list of simulations in the currently active study,
  2. a new PROD002 should appear in the list of Products in the user data.
  3. a new simulation new should appear in the list of study simulations
  4. a new central pane tab should open, with the details of the PROD002 product in it.

How does script running works

The way a script is run, is to gather a few key attributes from the running application into a Python dictionary, and execute the content of the script, passing these key attributes as the “known namespace”. For those familiar with Python, It is similar in spirit to the built-in function exec(), allowing one to do the following:

>>> exec("b = a+1", {"a": 3})
>>> print(b)
4

The dictionary passed as second argument to the exec() is often called the context inside which the expression "b = a+1" is executed. Since it contains a, the expression a+1 can be evaluated, and b will be set to 4. The main difference is that the context passed to run a Reveal script currently contains the following:

  • "app" giving access to the kromatography.app.krom_app.KromatographyApp object supporting the complete application (all windows), and to the job_manager among other things,
  • "user_datasource" giving direct access to the kromatography.model.data_source.SimpleDataSource which contains the user data.
  • "task" giving access to the kromatography.ui.tasks.KromatographyTask supporting the window from which the script is being run.
  • "study" giving direct access to the study contained in the window from which the script is being run.
  • "study_datasource" giving direct access to the currently active study’s data container.

So, as long the user script uses any of these variables (app, user_datasource, task, study, study_datasource) and import everything else, a valid script is defined and can be run.