Friday, October 15, 2010

abap program for Parallel processing: breaks a report to pieces and processes it parallel to reduce the run-time

REPORT ZPARPROC.
************************************************************************
* This program implements parallel processing. Many time consuming
* report can be broken into equal pieces and run parallel, reducing
* the runtime (in this case searching for a string in the source code
* of a set of abaps). The main program (mother) creates CHILD_NU
* number of child programs(1-9),gives each of them a separate task, runs
* them parallel and then collects the end-result. The child program is
* actually embedded in the mother program, so the mother virtually
* gives birth of her children. A child that finished it's part, commits
* suicide: deletes itself at the end.
* Here is the result of a performonce test with 1-9 children processes
* for a light and a heavy task:
*
* children# light(Z* programs)(sec) heavy(R* programs)(sec)
* 1 51 664 100%
* 2 28 357
* 3 22 250
* 4 20 232
* 5 18 207 31%
* 6 19 209
* 7 18 219
* 8 21 202
* 9 20 211
*
* Conclusions:
* The performance starts to deteriorate when the children# exceeds a
* certain, task specific number (the children start to compete
* against each other for the same resources)
* The heavier the task - the more significant the performance gain.
************************************************************************
* The parent's data declaration
TABLES: TRDIR, TBTCO, INDX.
PARAMETERS: CHILD_NU TYPE I DEFAULT '3',
PATTERN(2) DEFAULT 'Z%'.
DATA: C,
CHILD(32) VALUE 'ZCHILD#',
JOBNAME(32),
ABAP_NAME(8).
DATA: BEGIN OF IV_TABLE OCCURS 10, "Table of intervals
LOW LIKE TRDIR-NAME,
HIGH LIKE TRDIR-NAME,
END OF IV_TABLE.
DATA: BEGIN OF ITAB OCCURS 500, "Table of program names
LINE(72),
END OF ITAB.
DATA: BEGIN OF CHILD_NAME_TAB OCCURS 10, "Table of the children's names
L(7), LL(23),
END OF CHILD_NAME_TAB.
DATA: BEGIN OF RESULT_TAB OCCURS 100, "The result of a single child
NAME LIKE TRDIR-NAME,
END OF RESULT_TAB.
DATA: BEGIN OF END_RESULT_TAB OCCURS 100, "The table of the end-results
NAME LIKE TRDIR-NAME,
END OF END_RESULT_TAB.

* Evenly distribute the work between the children
* With EXEC SQL it would be nicer, but less generic
DATA: INDEX TYPE I.
DATA: LINE_NUM TYPE I.
DATA: INTERVAL TYPE I.
DATA: BEGIN OF RANGE OCCURS 1000,
NAME LIKE TRDIR-NAME,
END OF RANGE.
* Timestamp
GET TIME. WRITE:/ SY-UZEIT. SKIP.
* Timestamp
SELECT * FROM TRDIR WHERE NAME LIKE PATTERN. "Fill up the range table
IF NOT TRDIR-NAME IS INITIAL.
RANGE = TRDIR-NAME.
APPEND RANGE.
ENDIF.
ENDSELECT.
SORT RANGE BY NAME.
DESCRIBE TABLE RANGE LINES LINE_NUM.
INTERVAL = LINE_NUM / CHILD_NU. "This will be the length of the
"working range of a single child
INDEX = 0.
DO. "Create the table of distinct ranges: 1 for each child
INDEX = INDEX + 1.
READ TABLE RANGE INDEX INDEX.
IF SY-SUBRC <> 0.
EXIT.
ENDIF.
IV_TABLE-LOW = RANGE-NAME.
INDEX = INDEX + INTERVAL.
READ TABLE RANGE INDEX INDEX.
IF SY-SUBRC <> 0.
INDEX = LINE_NUM.
READ TABLE RANGE INDEX INDEX.
IV_TABLE-HIGH = RANGE-NAME.
APPEND IV_TABLE.
EXIT.
ENDIF.
IV_TABLE-HIGH = RANGE-NAME.
APPEND IV_TABLE.
ENDDO.

* Create the children
DATA: NUM.
DO CHILD_NU TIMES. "Loop on the children
NUM = SY-INDEX. "The index of the child
CLEAR ITAB. REFRESH ITAB.
READ REPORT SY-REPID INTO ITAB.
LOOP AT ITAB.
C = ITAB+1(1).
IF C <> '@'. "Give birth of a child:
DELETE ITAB. "Filter it out from mamy's belly
ELSE.
SHIFT ITAB LEFT BY 2 PLACES.
MODIFY ITAB.
ENDIF.
ENDLOOP.
*
LOOP AT ITAB. "Give the child identity and a task to work on
REPLACE '#' WITH NUM INTO ITAB.
READ TABLE IV_TABLE INDEX NUM.
REPLACE '&' WITH IV_TABLE-LOW INTO ITAB.
IF SY-SUBRC = 0.MODIFY ITAB.CONTINUE.ENDIF.
REPLACE '$' WITH IV_TABLE-HIGH INTO ITAB.
MODIFY ITAB.
ENDLOOP.
*
CHILD+6(1) = NUM. "At last: create the child
INSERT REPORT CHILD FROM ITAB.
*
JOBNAME = CHILD. "Create a distinct job name to run the child in bg
JOBNAME+8(8) = SY-DATUM.
JOBNAME+16(8) = SY-UZEIT.
*
CHILD_NAME_TAB = JOBNAME. "Save the name of my wonderful child
APPEND CHILD_NAME_TAB.
*
PERFORM CREATE_A_CHILD_JOB. "Start the child
ENDDO.

* Wait for my children to complete their task and get the result
DO.
LOOP AT CHILD_NAME_TAB.
SELECT * FROM TBTCO WHERE JOBNAME = CHILD_NAME_TAB.
ENDSELECT.
IF TBTCO-STATUS = 'F' OR TBTCO-STATUS = 'A'.
REFRESH RESULT_TAB.
IMPORT RESULT_TAB FROM SHARED BUFFER INDX(ST) ID CHILD_NAME_TAB-L.
LOOP AT RESULT_TAB. "Merge the results in one single table
END_RESULT_TAB = RESULT_TAB.
APPEND END_RESULT_TAB.
ENDLOOP.
DELETE CHILD_NAME_TAB.
ENDIF.
ENDLOOP.
IF SY-SUBRC <> 0. EXIT. ENDIF.
ENDDO.

* Hooray, all the children are finished
LOOP AT END_RESULT_TAB.
WRITE: END_RESULT_TAB.
ENDLOOP.

* Timestamp
GET TIME. SKIP. WRITE:/ SY-UZEIT.
* Timestamp

* Schedule the child to run in background
FORM CREATE_A_CHILD_JOB.
DATA: JOBCOUNT LIKE TBTCJOB-JOBCOUNT.
ABAP_NAME = CHILD.
CALL FUNCTION 'JOB_OPEN'
EXPORTING
JOBNAME = JOBNAME
IMPORTING
JOBCOUNT = JOBCOUNT.
CALL FUNCTION 'JOB_SUBMIT'
EXPORTING
JOBNAME = JOBNAME
JOBCOUNT = JOBCOUNT
AUTHCKNAM = SY-UNAME
REPORT = ABAP_NAME.
CALL FUNCTION 'JOB_CLOSE'
EXPORTING
JOBNAME = JOBNAME
JOBCOUNT = JOBCOUNT
STRTIMMED = 'X'.
ENDFORM.

* Here is the child template in the mothers tummy:

*@program zchild#.
*@tables: trdir, indx.
*@data: begin of itab occurs 500,
*@ line(72),
*@end of itab.
*@data: begin of result_tab occurs 100,
*@ name like trdir-name,
*@end of result_tab.
*@data: string(20) value 'loadi'.
*@
*@select * from trdir where name between
*@'&'
*@and
*@'$'.
*@ read report trdir-name into itab.
*@ loop at itab.
*@ if itab-line cs string.
*@ write: trdir-name.
*@ result_tab = trdir-name.
*@ append result_tab.
*@ exit.
*@ endif.
*@ endloop.
*@endselect.
*@* Export the result to shared buffer
*@export result_tab to shared buffer INDX(ST) ID 'ZCHILD#'.
*@* The child commits a suicide after it has finished it's task
*@delete report 'ZCHILD#'.

No comments:

Tutorials on SAP-ABAP

Adobe Interactive Forms Tutorials

Business Server Pages (BSP)

Userexits/BADIs

Web Dynpro for ABAP (Step by step procedure for web dynpro,Tutorials on Web Dynpro,)

ALV Tutorials

Blog Archive

goodsites