How do I write a data Cleansing Script in P#?

Open Develop in the PetroVisor Build Section. Click on Cleansing, the click on the +.

 

Don't forget to save your script.

In some workspaces, Cleansing Scripts are already loaded and easily run.

Each data cleansing script consists of one Cleansing part (e.g. "ESP Data Cleansing"), which contains all filters related to this part. Each filter can contain a set of conditions for one signal.

 

An sample Data Cleansing script:

Cleansing "Sample Cleansing Script 1"

 Filter "esp pump temperature" in "degC"
                  If "esp pump temperature" in "degC" <= -5 

                 Then        Null()
                         ElseIf "esp pump temperature" in "degC" < 0 and
"esp pump temperature" in "degC" > -5

                 Then 0.0
                           Else "esp pump temperature" in "degC"
                  End If
 End Filter

End Cleansing

This example refers to the signal "esp pump temperature", as can be seen in the second line. The first If statement filters for temperatures smaller than -5°C, which will be cleared. A cell value of "Null()" means an empty cell. These values won't be written to the database at all.

The Elseif statement filters for temperatures between 0 and -5°C, which are set to 0.0°C. These values (0.0°C) will be written to the database.

The Else statement, finally, states that all other values (leaving those >=0) will be kept as they are and written to the database.

Summarizing, the Sample Cleansing Script 1 checks if the signal esp pump temperature contains values bigger than 0°C. If they are slightly negative, they are cleansed to be 0.0 and if they are strongly negative, they are disregarded.

 

Another sample Cleansing script:

Cleansing "Sample Cleansing Script 2"
 Filter "oil production rate" in "m3/h"
                 If "oil production rate" in "m3/h" <= -5 or"oil production rate" 
in "m3/h" > 150

                 Then Null()
                         ElseIf "oil production rate" in "m3/h" < 0 and 
"oil production rate" in "m3/h" > -5

                 Then 0.0
                         Else "oil production rate" in "m3/h"
                 End If
 End Filter

End Cleansing

 

This script removes all values from the signal "oil production rate", which are less than or equal to -5 or greater than 150 and sets all values between 0 and -5 to 0.0 All other values are not cleansed.

When editing or writing a new cleansing script, care has to be taken that the conditions affecting a signal are not contradictory. Like "normal" P# scripts, cleansing scripts can be saved, loaded and deleted.

 

Note: Some rules are not available in data cleansing scripts, as they are hard-coded in PetroVisor's source code. These are:

  • Replace the string OPEN by 1
  • Replace the string CLOSED by 0
  • Replace the string UNDEFINED by 0