Refactor to support SAP Business Technology Platform (BTP, fka SAP Cloud Platform) ABAP Environment (Steampunk)

[gregorwolf]

After installing SAP Note 2684665 - Custom code checks for SAP Cloud Platform ABAP Environment the SAP Code Inspector (Transaction SCI) Check Variant with the following settings:

results in the following errors:

Let's start fixing this errors to get abap2xlsx running in Steampunk.

[larshp]

Any thoughts on how to handle the reports, dynpros and gui frontend stuff? Perhaps split the code into backend and examples?

[gregorwolf]

I've just tried another activation in 2008 and got this errors that might be easy to address:

Use of Domain FLAG is not permitted.
Use of Domain INT1 is not permitted.
Use of Domain INT2 is not permitted.
Use of Domain INT4 is not permitted.
Use of Domain STRING is not permitted.
Use of Domain TEXT31 is not permitted.
Use of Domain UUID is not permitted.
Use of Domain XFELD is not permitted.

Use of Table DD02L is not permitted.
Use of Table DD03L is not permitted.

and

Use of Data Element DSTRING is not permitted.
Use of Data Element DYNPROFLD is not permitted.
Use of Data Element FIELDNAME is not permitted.
Use of Data Element GUID_16 is not permitted.
Use of Data Element INTTYPE is not permitted.
Use of Data Element LVC_NOKEYC is not permitted.
Use of Data Element NUM1 is not permitted.
Use of Data Element SCRTEXT_L is not permitted.
Use of Data Element SCRTEXT_M is not permitted.
Use of Data Element SCRTEXT_S is not permitted.
Use of Data Element TABFDPOS is not permitted.

[attilaberencsi]

Hello Guys,
I pulled the master branch into an ABAP Cloud Trial environment. There are around 170 items about the above mentioned DDIC usage errors.
I copied some problematic standard domains in first round to ZA2XLSX_ < standard domain name > and replaced in ZEXCEL data elements and activated. So far so good, seems working. I would do the same for other problematic DDIC usage objects.
Do you agree with this approach ?
Are we strict in terms of naming, should I call instead the domains etc. ZEXCEL_<standard_ddic_name> ?
What are the rules to push back changes to the repo ?
So as expected result, at least activation and syntax errors should dissapear.
Thanks, Attila

[sandraros]

For information, the errors above are partly addressed by:

• Removing unnessssary DDIC dependencies #692 (FLAG, INT1, INT2, ...)
• package structure #695 (demos moved to a separate package)
• Modification of SY fields #704
• refactoring/extract OLE methods #717
• refactoring: use of cl_object_collection_iterator #880
• Strict SQL (see Lars below comment)
• refactor all MOVE statements #926

[larshp]

its not possible with a single codebase targeting 702 when it contains SELECTs, Steampunk applies strict SQL, strict SQL is not possible in 702(introduced in 7.40, SP05),

having a automatic upport in a repository is a possibility, alternatively refactor SELECTs to something else

[larshp]

there are the following outside of demos, most of these can be refactored to avoid the SELECT,

[gregorwolf]

Wow, the number of objects with warnings reduced dramatically. The only warning left for the main library is:

ZCX_EXCEL CLAS W OTR (Online Text Respository) texts are not supported in SAP Cloud Platform, ABAP Environment
ZCX_EXCEL CLAS W Object ZCX_EXCEL partly imported

How can we tackle this?

[larshp]

yea, so above is the import errors, not the activation errors. Activation should give more errors, like #926

[GramontABAPEnvironment]

Hi Lars,

can you please give an update with respect to Steampunk enablement?
Looks still promising?

We are waiting for this since you announced
https://blogs.sap.com/2022/01/03/steampunk-released-abap-api-first-draft-is-up/

Thanks & Regards,
Peter Walcz

[gregorwolf]

Hi Peter,

stop waiting and start contributing. If you have the urgent need help to get it done.

Best Regards
Gregor

[larshp]

yes, pull requests welcome 😄

https://blogs.sap.com/2022/01/03/steampunk-released-abap-api-first-draft-is-up/ is one tool to help the process, but its still lots of work, many issues to be tackled

[larshp]

class cl_xml_document

[sandraros]

For information, the list of 100 first syntax errors with the latest abap2xlsx version which includes the DDIC fixes (#1169); unfortunately, by error, I also imported the objects of the _not_cloud package because the abapGit Eclipse plugin currently doesn't permit to omit one package, just objects, and their package is not shown...):

Description                                                                                           Resource                                    Location
CLOSE is not allowed in the current ABAP language version.                                            zcl_excel_reader_2007.aclass                line 4075
DEFINITION LOAD is not valid in the current ABAP language version.                                    zcl_excel_style_cond.aclass                 line 7
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_autofilters.aclass                line 118
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 365
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 369
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 1217
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 1218
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 1346
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 1347
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 1369
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_converter.aclass                  line 1704
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 671
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 686
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 907
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 908
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 909
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 996
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 1132
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 1140
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 1145
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 1358
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_ole.aclass                        line 1393
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_reader_2007.aclass                line 4062
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_table.aclass                      line 147
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_table.aclass                      line 198
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_worksheet.aclass                  line 1462
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_worksheet.aclass                  line 1469
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_worksheet.aclass                  line 3102
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_worksheet.aclass                  line 3346
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 3589
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 3611
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 4774
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 4784
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 4794
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 4804
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 4817
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 4841
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 4885
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 5158
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 5164
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 5170
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007.aclass                line 5176
DESCRIBE is not allowed in the current ABAP language version.                                         zcl_excel_writer_2007implementations.acinc  line 845
EXPORT is not allowed in the current ABAP language version.                                           zcl_excel_converter.aclass                  line 1785
GET is not allowed in the current ABAP language version.                                              zcl_excel_converter.aclass                  line 413
IMPORT is not allowed in the current ABAP language version.                                           zcl_excel_converter.aclass                  line 422
LOAD is not valid in the current ABAP language version.                                               zcl_excel_reader_huge_file.aclass           line 33
MESSAGE is not allowed in the current ABAP language version.                                          zcl_excel_converter.aclass                  line 281
MESSAGE is not allowed in the current ABAP language version.                                          zcl_excel_converter.aclass                  line 314
MESSAGE is not allowed in the current ABAP language version.                                          zcl_excel_converter.aclass                  line 1683
OPEN is not allowed in the current ABAP language version.                                             zcl_excel_reader_2007.aclass                line 4063
READ is not allowed in the current ABAP language version.                                             zcl_excel_reader_2007.aclass                line 4070
SEARCH is not allowed in the current ABAP language version.                                           zcl_excel_ole.aclass                        line 1327
SEARCH is not allowed in the current ABAP language version.                                           zcl_excel_ole.aclass                        line 1333
SET is not allowed in the current ABAP language version.                                              zcl_excel_ole.aclass                        line 1373
SET is not allowed in the current ABAP language version.                                              zcl_excel_ole.aclass                        line 1377
SET is not allowed in the current ABAP language version.                                              zcl_excel_ole.aclass                        line 1392
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_ole.aclass                        line 1374
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_ole.aclass                        line 1397
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_ole.aclass                        line 1400
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_ole.aclass                        line 1413
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_ole.aclass                        line 1416
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_ole.aclass                        line 1461
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_ole.aclass                        line 1465
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_worksheet.aclass                  line 1339
WRITE is not allowed in the current ABAP language version.                                            zcl_excel_writer_csv.aclass                 line 244
Field "FIELDCAT" is unknown.                                                                          zcl_excel_worksheet.aclass                  line 3982
Incorrect nesting: For the statement "ENDMETHOD", there is no open structure introduced by "METHOD".  zcl_excel_writer_csv.aclass                 line 275
Incorrect nesting: For the statement "ENDSELECT", there is no open structure introduced by "SELECT".  zcl_excel_writer_csv.aclass                 line 114
Method "GET_INDEX" is unknown or PROTECTED or PRIVATE.                                                zcl_excel_writer_2007.aclass                line 2555
Method "GET_INDEX" is unknown or PROTECTED or PRIVATE.                                                zcl_excel_writer_2007.aclass                line 6317
Method "GET_INDEX" is unknown or PROTECTED or PRIVATE.                                                zcl_excel_writer_2007.aclass                line 6341
Method "GET_INDEX" is unknown or PROTECTED or PRIVATE.                                                zcl_excel_writer_2007.aclass                line 6365
Method "GET_INDEX" is unknown or PROTECTED or PRIVATE.                                                zcl_excel_writer_2007.aclass                line 6424
Method "GET_INDEX" is unknown or PROTECTED or PRIVATE.                                                zcl_excel_writer_2007.aclass                line 6448
Method "GET_INDEX" is unknown or PROTECTED or PRIVATE.                                                zcl_excel_writer_2007.aclass                line 6472
Method "GET_MEDIA_NAME" is unknown or PROTECTED or PRIVATE.                                           zcl_excel_writer_2007.aclass                line 2571
Method "GET_POSITION" is unknown or PROTECTED or PRIVATE.                                             zcl_excel_writer_2007.aclass                line 6301
Method "GET_POSITION" is unknown or PROTECTED or PRIVATE.                                             zcl_excel_writer_2007.aclass                line 6325
Method "GET_POSITION" is unknown or PROTECTED or PRIVATE.                                             zcl_excel_writer_2007.aclass                line 6349
Method "GET_POSITION" is unknown or PROTECTED or PRIVATE.                                             zcl_excel_writer_2007.aclass                line 6408
Method "GET_POSITION" is unknown or PROTECTED or PRIVATE.                                             zcl_excel_writer_2007.aclass                line 6432
Method "GET_POSITION" is unknown or PROTECTED or PRIVATE.                                             zcl_excel_writer_2007.aclass                line 6456
Only reads can be performed on the field "SY" in the current ABAP language version.                   zcl_excel_commontestclasses.acinc           line 26
Only reads can be performed on the field "SYST" in the current ABAP language version.                 zcx_excel.aclass                            line 16
The use of Class C_OI_CONTAINER_CONTROL_CREATOR is not permitted.                                     zcl_excel_ole.aclass                        line 270
The use of Class C_OI_ERRORS is not permitted.                                                        zcl_excel_oleimplementations.acinc          line 13
The use of Class C_OI_ERRORS is not permitted.                                                        zcl_excel_oleimplementations.acinc          line 13
The use of Class CL_ABAP_CONV_IN_CE is not permitted.                                                 zcl_excel_reader_2007testclasses.acinc      line 31
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_converter_alv_grid.aclass         line 35
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_converter_alv_grid.aclass         line 38
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_ole.aclass                        line 386
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_oledefinitions.acinc              line 36
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_oledefinitions.acinc              line 42
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_oleimplementations.acinc          line 101
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_worksheet.aclass                  line 150
The use of Class CL_GUI_ALV_GRID is not permitted.                                                    zcl_excel_worksheet.aclass                  line 932
The use of Class CL_GUI_CONTAINER is not permitted.                                                   zcl_excel_ole.aclass                        line 281
The use of Class CL_IXML is not permitted.                                                            zcl_excel_reader_2007.aclass                line 668
The use of Class CL_IXML is not permitted.                                                            zcl_excel_reader_2007.aclass                line 668

[sandraros]

its not possible with a single codebase targeting 702 when it contains SELECTs, Steampunk applies strict SQL, strict SQL is not possible in 702(introduced in 7.40, SP05),

having a automatic upport in a repository is a possibility, alternatively refactor SELECTs to something else

I agree to go for a new repository for Cloud.

It would be a new abap2xslsx with restricted features (e.g. zcl_excel_font cannot work because it relies on old SAPscript/Smart Form stuff).

abap2xlsx has several flaws too, many methods for the same thing, heterogeneous logic, public members which should be read-only or private, public instantiation while it should be private, etc.

I guess that @attilaberencsi would like to be part of it (did the last PR for DDIC refactoring #1169).

If you agree, then which version should we target, how to start a draft without making it public yet, should we first have all objects and then refactor one thing at a time, which name for the repository, etc.? (no idea at all as it is my first collaboration on a new GitHub project)

NB: I started working on it in https://github.com/sandraros/abap2xlsx-cloud/tree/draft.

[larshp]

Suggest:
1: get the work already done in ABAP cloud direction merged, ie. #1125
2: need to get unit testing set up running automatically, this allows us to refactor, ie. #872
3: make as much code compatible in the single repository as possible
4: automatic stuff, more repos etc, but this is some time in the future

zcl_excel_font this class can exist and run in ABAP Cloud, having just one source to maintain

[attilaberencsi]

Tried to relink it to the new cloud repo, but the code from Sandras @sandraros abap2xlsx-cloud repo was not taken over. Not sure about the reason. Whether this is normal, when we have objects never activated because of syntax issues, and we try to pull again some changes by others from GitHub froma new repo.
Therefore I removed the on-premise abap2xlsx repo from the US Trial BTP ABAP Environment completely,so that when needed the new cloud version can be pulled.

The ideal would be if we could co-work on the same code base right in the system.
One case: I adjusted the zcx_excel exception object to make it working in the cloud. That step was OK, activated successfully. After I started adjusting the usages to use new the cloud-compliant version, but those classes contain plenty of other issues, which lead to plenty of another issues in a chain. So finally I could not activate the classes using zcx_excel, just save after fixing the exception handling. So one topic like exception handling would be OK at this stage everywhere in the repo, despite the code of usages is still syntactically incorrect due other topics to be adressed. Adding more and more fixes on dependent objects to be able to activate them would involve adding neverending number of objects to the commit of exception handling, and since we work on separate topics this could lead later to high merging costs due we work in a separate system.

Can You please advise what is the best to do in this case ?
May we commit syntactically incorrect objects, is it a correct procedure in this case ?
Not sure that possible at all to do that with ABAPGit, if the object is not activated.

Thank You, Attila

[sandraros]

Just saw that now. Yes, the inactive objects are not pushed. I continued working on modifying the objects, activated many of them, and pushed again in my repository. I think it's impossible to push inactive objects, and impossible to force the activation of invalid objects.

In fact, I don't know which approach to use. Lars approach is too complex for me, just because I don't know the stuff. By spending a few hours on it, I wanted to understand exactly what is to be done, even if we choose another solution.

There are three important topics to convert to Cloud without redesigning anything:

• replace IXML. My stupid approach was to keep abap2xlsx code as it is and create a Z wrapper of SXML with a Z IXML-like interface, currently it's the interface without the implementation.
• zcl_excel_drawing uses non-released stuff.
• zcl_excel_font uses SAPscript/Smart Font stuff, a new solution must be implemented to work with fonts.

I need a few hours to complete. After that, it will be more clear for me and I can discuss. No problem to restart from scratch using any better approach.

[larshp]

😒

you are both at step 4 in my suggestion, do the easy stuff first, keeping compatibility

the "SAP" approach is re-write everything. I think the worst approach is to re-write everything plus doing double maintenance

[sandraros]

@larshp okay I agree with your approach and I'll follow it. I was afraid of parts 2 and 4 (also upport you mentioned on Nov 19, 2021, maybe part 4).

I propose to first create a branch "cloud" where to work and do the PRs.

Concerning the points of your approach:

1. Review existing Cloud PRs. Okay to check it first.
2. Run automatically unit testing. I don't know what it's about.
3. Let's do a short list of topics to split the work (announce what you pick up to avoid double work then do a pull request). e.g.
   • basic stuff like replacing describe table x lines y with y = lines( x ), seoclsname with ``, data element flag with `abap_boolean` (if it exists in 7.02?), etc.
   • rework IXML
   • rework "drawing" class
   • rework "font" class
   • etc.
4. automatic stuff. What is it? Is it upporting e.g. converting 7.02 code to Cloud code like select ... into x into select ... into @x?

NB: concerning IXML, I have isolated my yesterday work (creation of all empty interfaces, classes, attributes and methods needed by abap2xlsx) in this repository/branch.

[sandraros]

I'm proposing to work now on replacing IXML classes/interfaces (not supported in ABAP Cloud) with equivalent ZCL_EXCEL*/ZIF_EXCEL* classes/interfaces which will use internally SXML (supported in ABAP Cloud). I'll do a PR which we can discuss. Please tell me if anybody has already started working on it, or proposes another solution. You can find the latest work here.

[sandraros]

FYI, I guess I need one more week to finish the main work at https://github.com/sandraros/ixml-cloud/tree/needed-by-abap2xlsx. After that, I will need a few more days to propose a PR at https://github.com/abap2xlsx/abap2xlsx.

[sandraros]

@attilaberencsi It's done, #1188: IXML is replaced with SXML (see PR description for more information). I tested it in a 7.52 system, not in a Cloud system yet. All test classes pass, also ZDEMO_EXCEL_CHECKER (new version needed in abap2xlsx/demos#51). Could you review it please?
I have also added everyone possible to review it, if available. @larshp @AndreaBorgia-Abo @bigld.
I'm open to any suggestion, even rewriting it. Thank you!
@larshp I have one issue with https://github.com/abap2xlsx/abap2xlsx/actions/runs/7899225079/job/21558215984, should I do something?

[AndreaBorgia-Abo]

Cool, I'll try it at work on a 7.31 system and in the weekend on my MiniSAPs.

[attilaberencsi]

I tried to pull to in the latest BTP ABAP Trial Environment (US East). Someone else already pulled and deleted it in package YABAP2XLSX on 2nd of January, w/o releasing the TR, so we have some junks in TADIR which is not good when I pull. I have to try to clean it up first or take over the there.

[gregorwolf]

Hi @attilaberencsi ,

do you have a possibility to get a BTP Free Tier? There you would have you own ABAP Enviornment and you will not run into this issues.

[larshp]

thing is, IXML is supported in cloud, https://abapedia.org/steampunk-2305-api/if_ixml_document.intf.html needs some workarounds tho, guessing 100 or 200 lines of code will do

[sandraros]

Too bad, better late than never... 😓
Without me on IXML. Any volunteer to do this part?

• iXML Library for ABAP Cloud
• CL_IXML_CORE instead of CL_IXML
• All stream, parser and renderer interfaces now suffixed CORE
• The other IF_IXML_* interfaces are unchanged

[attilaberencsi]

Hi @gregorwolf,
yes right this is a good dolution.
I have one free tier on my credit card already, due corporate things I cannot expose it to the community unfortunately . My idea was that better to do it on trial which is public and everyone has access and can contribute. Using abapGit on code directly in the system having bunch of syntax issues is not really working to publish it back to the repo. If we could work on it in the cloud, we could also immediately see, what is released what is not and what the others doing and what are the remaining syntax issuues. Working in this situation on separate islands is not really efficient, and I hardly track it. I started to fix the tadir issues, will be ready soon.

[gregorwolf]

Hi @attilaberencsi,

I've sent you an invite to the abapGit Slack so that we can have a private chat. Let's try something on my Free Tier ABAP Environment.