Hi,

I am starting with the first blog.

Recently, I have come across a requirement where we are supposed to integrate with MinIO. MinIO is a High-Performance Object Storage released under GNU Affero General Public License v3.0.

It is API-compatible with the Amazon S3 cloud storage service. It can handle unstructured data such as photos, videos, log files, backups, and container images with a current maximum supported object size of 50TB.

We had a requirement to upload and download files to/from MinIO bucket. This will cover both functionalities in single blog.

In the Upload scenario, we used to get PDF files from SAP System as attachment to proxy which was extracted in Java mapping and sent to MinIO bucket.

Source structure,

<?xml version=”1.0″ encoding=”utf-8″?>

<n0:MT_ProxyminioSender xmlns:n0=” ” xmlns:prx=”urn:sap.com:proxy:ECP:/1SAI/TAS8E5273429B1BFFB79A16:756″>

<DOCNO>5400046098</DOCNO>

<Filename>5400046098_1.PDF</Filename>

</n0:MT_ProxyminioSender>

Here with DOCNO, we are creating a folder and filename will be referred from Filename field.

Here we are using parameterized mapping to get the URL, bucket, secret key and access key.

Java mapping can be downloaded from below link,

link

Also pasting code here,

import java.io.ByteArrayInputStream;

import java.io.ByteArrayOutputStream;

import java.io.InputStream;

import java.io.OutputStream;

import java.net.URL;

import java.nio.charset.StandardCharsets;

import java.util.HashMap;

import java.util.Map;

import java.util.concurrent.TimeUnit;

import com.sap.aii.mapping.api.*;

import io.minio.BucketExistsArgs;

import io.minio.GetPresignedObjectUrlArgs;

import io.minio.MinioClient;

import io.minio.PutObjectArgs;

import io.minio.http.Method;

import javax.xml.parsers.DocumentBuilder;

import javax.xml.parsers.DocumentBuilderFactory;

import org.w3c.dom.Document;

import org.w3c.dom.Element;

import org.w3c.dom.Node;

import org.w3c.dom.NodeList;

import com.sap.engine.interfaces.messaging.api.auditlog.*;

import com.sap.engine.interfaces.messaging.api.exception.MessagingException;

import com.sap.engine.interfaces.messaging.api.*;

public class GenericFileUploadtoMinio extends AbstractTransformation{

@Override

public void transform(TransformationInput transformationInput, TransformationOutput transformationOutput) throws StreamTransformationException {

MessageKey key = null;

AuditAccess audit = null;

final String DASH = “-“;

String msgID=transformationInput.getInputHeader().getMessageId();

String uuidTimeLow = msgID.substring(0, 8);

String uuidTimeMid = msgID.substring(8, 12);

String uuidTimeHighAndVersion = msgID.substring(12, 16);

String uuidClockSeqAndReserved = msgID.substring(16, 18);

String uuidClockSeqLow = msgID.substring(18, 20);

String uuidNode = msgID.substring(20, 32);

String msgUUID =   uuidTimeLow + DASH + uuidTimeMid + DASH + uuidTimeHighAndVersion + DASH  + uuidClockSeqAndReserved + uuidClockSeqLow + DASH + uuidNode;

try {

audit = PublicAPIAccessFactory.getPublicAPIAccess().getAuditAccess();

} catch (MessagingException e1) {

// TODO Auto-generated catch block

e1.printStackTrace();

}

key = new MessageKey(msgUUID, MessageDirection.OUTBOUND);

try {

getTrace().addInfo(“Java Mapping Program Started!”);

String DocNumber = “”;

String strFileName = “”;

String strDirectory = “”;

String dir = “”;

String slash = “/”;

String colon = “:”;

//Get the handler for processing the attachments of source message

InputAttachments attachment = transformationInput.getInputAttachments();

String url = transformationInput.getInputParameters().getString(“url”);

String bucketName = transformationInput.getInputParameters().getString(“bucketname”);

String accesskey = transformationInput.getInputParameters().getString(“accesskey”);

String secretkey = transformationInput.getInputParameters().getString(“secretkey”);

String nodename = transformationInput.getInputParameters().getString(“nodename”);

String folderpath = transformationInput.getInputParameters().getString(“folderpath”);

String auth = “?accessKey=”+accesskey+”&secretKey=”+secretkey;

InputStream inputstream = transformationInput.getInputPayload().getInputStream();

OutputStream outputstream = transformationOutput.getOutputPayload().getOutputStream();

//an instance of factory that gives a document builder

DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();

//an instance of builder to parse the specified xml file

DocumentBuilder db = dbf.newDocumentBuilder();

Document doc = db.parse(inputstream);

doc.getDocumentElement().normalize();

getTrace().addInfo(“XML Read Started”);

NodeList list = doc.getElementsByTagName(nodename);  //”n0:MT_ProxySender”

for (int temp = 0; temp < list.getLength(); temp++) {

Node node = list.item(temp);

if (node.getNodeType() == Node.ELEMENT_NODE) {

Element element = (Element) node;

// get value for RFQNumber

DocNumber = element.getElementsByTagName(“DOCNO”).item(0).getTextContent();

getTrace().addInfo(“DOCNO READ” +DocNumber );

// get value for Filename

strFileName = element.getElementsByTagName(“Filename”).item(0).getTextContent();

getTrace().addInfo(“File Name READ” +strFileName );

}

}

String ip = “INPUT”;

String path = folderpath+slash+DocNumber+slash+strFileName; //”PO”; //DocNumber+slash+ip+slash+strFileName;

//Fetch the collection of attachments in the source message

for (String id : attachment.getAllContentIds(false)) {

//Read the content of the attachment and assign the content to output

outputstream.write((attachment.getAttachment(id)).getContent());

}

//Code to convert OutputStream to InputStream

ByteArrayOutputStream bos = (ByteArrayOutputStream)outputstream;

//            ByteArrayInputStream inStream = new ByteArrayInputStream( bos.toByteArray());

InputStream input = new ByteArrayInputStream( bos.toByteArray());

//            byte[] bytes = inStream.toString().getBytes(StandardCharsets.UTF_8);

//            InputStream input = new ByteArrayInputStream(bytes);

audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS, “AccessKey “+accesskey+” SecretKey “+secretkey);

MinioClient s3Client =

MinioClient.builder()

.endpoint(url)

.credentials(accesskey, secretkey)

.build();

boolean found =

s3Client.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());

if(found){

audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS, “Bucket Found Successfully!!”);

getTrace().addDebugMessage(“Bucket ” +bucketName+ ” found Successfully”);

s3Client.putObject(

PutObjectArgs.builder().bucket(bucketName).object(path).stream(

input, -1, 10485760)

.contentType(“application/pdf”)

.build());

audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS, “200 – File Uploaded Successfully!!”);

//GET Created File URL

Map<String, String> reqParams = new HashMap<String, String>();

reqParams.put(“response-content-type”, “application/json”);

audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS, “Path “+path);

String urlString = s3Client.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder()

.method(Method.GET)

.bucket(bucketName)

.object(path)

.expiry(1, TimeUnit.HOURS)

.extraQueryParams(reqParams)

.build());

audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS, “FileURL “+urlString);

urlString = urlString.replace(“%2F”,”/”);

URL       endpointUrl = new URL(urlString);

// Create Dynamic Configuration Object

DynamicConfiguration conf = transformationInput.getDynamicConfiguration();

String urlvar = endpointUrl.toString();

urlvar = urlvar.replace(“%2F”,”/”);

DynamicConfigurationKey key1 = DynamicConfigurationKey.create( “http:/”+”/sap.com/xi/XI/System/REST”,”endpoint”);

conf.put(key1,urlvar);

}else {

getTrace().addDebugMessage(“Bucket ” +bucketName+ ” not found”);

}

}catch(Exception e) {

getTrace().addDebugMessage(e.getMessage());

audit.addAuditLogEntry(key, AuditLogStatus.ERROR, “Exception Raised: “+e.toString());

throw new StreamTransformationException(e.toString());

}

}

}

Here after file upload we are getting endpoint URL string where file can be located which will be passed as a dynamic parameter to receiver channel URL

The receiver channel config is as below,

Receiver channel_1

Receiver channel_2

Receiver channel_3

In the Download scenario, we created UDF where filenames were provided from source structure. Perform the below steps,

1. Search for file in MINIO bucket
2. If found, Extract file data
3. Attach to receiver proxy to SAP

Mapping step,

Mapping of UDF

DownloadFileUDF

UDF Code is as below,

public String AttachFileFromMinIO(String bucketName, String accesskey, String secretkey, String url, String attachments, String tenderno, Container container) throws StreamTransformationException{

String[] arrOfStr = attachments.split(“;”);

String slash = “/”;

String ip = “INPUT”;

String op = “OUTPUT”;

int DEFAULT_BUFFER_SIZE = 8192;

String attach = “”;

String filename = “”;

try{

getTrace().addInfo(“————-AttachFileFromMinIO Started—————–“);

// Write attachment

GlobalContainer globalContainer = container.getGlobalContainer();

OutputAttachments outputAttachments = globalContainer.getOutputAttachments();

MinioClient s3Client =

MinioClient.builder()

.endpoint(url)

.credentials(accesskey,secretkey)

.build();

OutputStream out = new ByteArrayOutputStream();

//getTrace().addInfo(“Minio Bucket Found ” +s3Client.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build()));

for(int i =0; i<=arrOfStr.length; i++)

{

try{

getTrace().addInfo(“arrOfStr ” +arrOfStr[i]);

attach = attach+”,”+arrOfStr[i];

filename = arrOfStr[i];

String path = tenderno+slash+op+slash+filename;

getTrace().addInfo(“Filename ” +filename);

try ( InputStream istream = s3Client.getObject(

GetObjectArgs.builder()

.bucket(bucketName)

.object(path)//”OUTPUT/MinIOTest.pdf”

.build())) {

// Read data from stream

getTrace().addInfo(“File “+filename+” downloaded”);

byte[] bytes = IOUtils.toByteArray(istream);

istream.close();

Attachment newAttachment = outputAttachments.create(filename,”application/pdf”, bytes);

outputAttachments.setAttachment(newAttachment);                                                              }

}catch(Exception e){

getTrace().addInfo(“Error while processing file ” +filename);

}

}

}catch(Exception e){

getTrace().addInfo(“Error while processing file”);

}

return attachments;

}

DownloadFileOutput

We came across the blog https://blogs.sap.com/2020/10/20/file-upload-in-aws-s3-using-rest-api/ which was using AWS S3 library but here I have referred to libraries provided by MinIO (https://min.io/docs/minio/linux/developers/java/API.html)

I hope this blog is helpful for you all while integrating MinIO system with SAP.

If so, click on “like” or “share”. I’m looking forward to your feedback and thoughts.

Thank you.