Multi-threading as a method to accelerate certain procedures

Publications: Multi-threading as a method to accelerate certain procedures
1С:Enterprise 8 platform has the necessary tools to provide a multi-threading, but in practice these tools are undeservedly forgotten.

Of course, multi-threading is not a panacea, but this is a good way to:

  1. reduce the duration of executing the time-consuming procedures (in some cases – by the orders of magnitude!);
  2. increase the utilization of equipment resources.

Not every time-consuming procedure can be performed in the multi-threaded mode, but only those that operate with a big number of non-interconnected information blocks. A couple of examples:

  1. there are the hundreds of shops operating with self-written software and a single base in the head company where the sales transactions are daily dumped. Loading is initiated by the user with a button and takes a long time, because there are lots of shops and transactions, and the data packages are processed sequentially. In this situation, the sales of each shop do not depend on each other, therefore, the parsing of files and forming the documents in the system can be performed in the multi-threaded mode. But the document conduction will be most likely a bottleneck and must be performed in a single-threaded mode. In this example, the acceleration can be not very high, since a formation of documents takes less time than their conduction.
  2. There is a big database with sales transactions of the shops (from previous example) and there is a necessity to dump periodically these data in the third-party BI-system using the tools of 1С. The dumping is initiated by the user and takes a long time, because the system needs to get from database a millions of rows and dump them in the intermediate base for BI. During data dumping the equipment is loaded unissentially. In this situation, the data dumping could be parallelized – dump data simultaneously over the different shops. This will significantly accelerate the process and will allow feeling in full an effect of a powerful hardware.

Built-in language tools to perform multi-threaded procedure.

It is possible to organize the most effective multi-threading using the Background jobs (do not confuse with the scheduled jobs). It is required to form in cycles the portions of background jobs and wait their completion. Code example for the second example (dumping a large data array by portions):
1) Procedure for initiating the multi-threaded execution of code:

Procedure PressExecuteButton(Button)

    //specifies the number of threads to be run simultaneously
    NumberConcurrentThreads = 10;

    ArrayJobs = New Array;

    Query = New Query(
    "SELECT DISTINCT
    |    StoredGoods.Warehouse
    |FROM
    |    AccumulationRegister.StoredGoods AS StoredGoods
    |WHERE
    |    StoredGoods.Perion BETWEEN &Date1 AND &Date2");
    Query.SetParameter("Date1", BeginDate);
    Query.SetParameter("Date2", EndDate);

    Result = Query.Execute().Unload();

    For Each Str In Result Do

        ArrayParameters = New Array;
        ArrayParameters.Add(BeginDate);
        ArrayParameters.Add(EndDate);
        ArrayParameters.Add(Str.Warehouse);

        Job = BackgroundJobs.Execute("DataDumpingOnServer.UnloadDataOnParties", ArrayParameters);

        ArrayJobs.Add(Job);

        If ArrayJobs.Count() >= NumberConcurrentThreads Then
            Try
                BackgroundJobs.WaitForCompletion(ArrayJobs);
            Except
            EndTry;
            ArrayJobs.Clear();
        EndIf;

    EndDo;

    If ArrayJobs.Count() > 0 Then
        Try
            BackgroundJobs.WaitForCompletion(ArrayJobs);
        Except
        EndTry;
        ArrayJobs.Clear();
    EndIf;

    Message("Duration of the procedure - " + (CurrentDate() - BeginTime) + " sec.");

2) Procedure that directly performs a background job (main logic):
A general module «DataDumpingOnServer» executed on the server:

Procedure UnloadDataOnParties(BeginDate, EndDate, Warehouse) Export

    Query = New Query(
    "SELECT
    |    *
    |FROM
    |    AccumulationRegister.StoredGoods AS StoredGoods
    |WHERE
    |    StoredGoods.Period BETWEEN &Date1 И &Date2
    |    AND StoredGoods.Warehouse = &Warehouse");

    Query.SetParameter("Date1", BeginDate);
    Query.SetParameter("Date2", EndDate);
    Query.SetParameter("Warehouse", Warehouse);

    Result = Query.Execute().Unload();

    For Each Str In Result Do
        //Do something with data...
    EndDo;
...

In this case, it is required to take into account that the choice of the number of parallel jobs need to be taken responsibly – even if to launch a few dozen of them, there is a high probability to «hang» both the application server and the DBMS server.

Click to rate this post!
[Total: 0 Average: 0]

Leave a Reply

Your email address will not be published. Required fields are marked *