How do I write a data Cleansing Script in P#?

Open the P# Editor in the PetroVisor Workflow Automation Tool. Click on Options and select Cleansing Scripts in the Mode.

 
  • To edit an existing data cleansing script, click on the Load button in the menu bar, select one of the available scripts and click Load to open the selected script.
  • To write a new data cleansing script, click New in the menu bar. Don’t forget to Save your new data cleansing script when its finished.

Each data cleansing script consists of one Cleansing part (e.g. "ESP Data Cleansing"), which contains all filters related to this part. Each filter can contain a set of conditions for one signal.

 

An sample Data Cleansing script:

Cleansing "Sample Cleansing Script 1"

 Filter "esp pump temperature" in "degC"
                  If "esp pump temperature" in "degC" <= -5 

                 Then        Null()
                           ElseIf "esp pump temperature" in "degC" < 0 and "esp pump temperature" in "degC" > -5 

                 Then 0.0
                           Else "esp pump temperature" in "degC"
                  End If
 End Filter

End Cleansing

This example refers to the signal "esp pump temperature", as can be seen in the second line. The first If statement filters for temperatures smaller than -5°C, which will be cleared. A cell value of "Null()" means an empty cell. These values won't be written to the database at all.

The Elseif statement filters for temperatures between 0 and -5°C, which are set to 0.0°C. These values (0.0°C) will be written to the database.

The Else statemtnent, finally, states that all other values (leaving those >=0) will be kept as they are and written to the database.

Summarizing, the Sample Cleansing Script 1 checks if the signal esp pump temperature contains values bigger than 0°C. If they are slightly negative, they are cleansed to be 0.0 and if they are strongly negative, they are disregarded.

 

Another sample Cleansing script:

Cleansing "Sample Cleansing Script 2"
 Filter "oil production rate" in "m3/h"
                 If "oil production rate" in "m3/h" <= -5 or"oil production rate" in "m3/h" > 150

                 Then Null()
                         ElseIf "oil production rate" in "m3/h" < 0 and "oil production rate" in "m3/h" > -5

                 Then 0.0
                         Else "oil production rate" in "m3/h"
                 End If
 End Filter

End Cleansing

This script removes all values from the signal "oil production rate", which are less than or equal to -5 or greater than 150 and sets all values between 0 and -5 to 0.0 All other values are not cleansed.

When editing or writing a new cleansing script, care ha to be taken that the conditions affecting a signal are not contradictory. Like "normal" P# scripts, cleansing scripts can be saved, loaded and deleted.

 

Note: Some rules are not available in data cleansing scripts, as they are hard-coded in PetroVisor's source code. These are:

  • Replace the string OPEN by 1
  • Replace the string CLOSED by 0
  • Replace the string UNDEFINED by 0