E2E Bridge E2E Commerce

Changes in Persistent state object deletion – Runtime 2019.1

E2E Forum News Changes in Persistent state object deletion – Runtime 2019.1

Viewing 1 post (of 1 total)
  • Author
  • #3235

    In Runtime 2019.1 we have optimized the last stage of Persistent State object lifetime – the deletion / cleanup phase.

    Runtime before 2019.1

    The cleanup of an Persistent State object is an important step that keeps the size of the database manageable and ensures continuous good performance of the Persistent State machinery. Many SQL statements need to be executed in order to remove data from all associated tables – so this is also a relatively expensive step.

    Up until Runtime 2019.1, the process has been as follows:

    1. Perform the final transition – in transaction log this is the transition to “AnonymousFinalState”
    2. Internally change the state to “- -8<- -” and sent the “FINALIZE” signal – this is why one could sometimes see this “funny” state in the Console.
    3. Process the “FINALIZE” signal with the same rules as a regular signal (so also in the same order), but instead of any transition:
      1. run default handler for undelivered signals (this step turned out not to work reliably)
      2. delete the object, all its tokens, signals, conversations, history states, etc.

    This behavior put additional load on the Persistent State engine.

    • For once we had to do more cleanup – an object in the final state has already been dead for all intents and purpose, so all but one token must had been removed along with their events.
    • Then another event had to be inserted into event queue and later processed like any other event.
    • At the end there had been a cleanup phase where the default handler for undelivered signals had been called and then everything connected to the object deleted from the database.

    One problem with that approach is that some very similar DELETE queries are run. DELETE queries tend to be slow and have higher potential for creating deadlocks. Second problem is the additional transition that adds load to the standard event queue (additional signal, worker).
    And then there’s the default signal handler. First, it had been unreliable – it would fire only for events that came at the right time and even then the handler would get the object ID of an de facto dead object which may be very unexpected and limits the usefulness of calling the handler at that stage.

    The more subtle problem had manifested itself in transaction log. The object deletion had been an internal transition and hadn’t been logged. With larger counts of objects this had caused “holes” in transaction log – periods when the Persistent State engine seemingly had been doing nothing. This had been very confusing for anyone trying to analyze Persistent State performance.

    To sum up, the approach had been slow and had reliability problems.

    Runtime 2019.1+

    The new implementations simplify things a lot. Since Runtime 2019.1 Persistent State objects (and all associated database records) are deleted upon reaching the final state.

    The moment the object begins the final transition it is considered dead. If an event manages to come in after begin of the final transition, that event is lost. The same is true for the events that are already in the database but queued after the one that caused the transition.
    In this case, however, a warning will be logged upon object deletion to draw attention a possible problem and modeller should check why other parts of the model still consider the object alive.


    In an artificial test, designed only to measure the deletion performance, Runtime 2018.12 needed 18m 55s to complete (and had a gap of 3m 34s in transaction log), where Runtime 2019.1 reached 13m 55s. That represents 26.4% improvement in that regard.

    The update to Runtime 2019.1 is therefore especially recommended for all cases where one would expect high concentration of objects transitioning to their final state.

Viewing 1 post (of 1 total)
  • You must be logged in to reply to this topic.