Custom EDIFACT file

I'm going to upgrade a BizTalk 2002 solution to BizTalk 2006 (or maybe BizTalk 2006 R2 if it makes things easier) and wonder if there are some ways to simplify the solution.
We are using BizTalk 2002 to import data from custom EDIFACT files into a SQL database. The files includes up to 6 different record types or header information or trailer information.

A record looks like this (where the first 4 digits are the number of characters in the record):
0298VKD+103:59909247:3::TKVKD:8:1+20040519:163829247'VAI+I:KK+CH:1234567'MGA+M:EEME+1+1+20040519'MGA+O: O+1+1+20040504'VAK+I:KKIIII+TKN:4+ABCD Efghij Klmn+Shs'FMN+I:IIII+T1:333++2'KAP+M:KEEME+1+11394000++20040518+2'KAP+O: O+1+11394000++20040503+2'ZAZ+M:MM+1284050+20040519'ZAZ+O:AA+1255260+20040504'

We have made a pre-processor to split the different record types into separate files which are picked up by separate file receive functions. (The records are not separated with any characters. We are using the first 4 digits to split them). It also replaces empty fields (++) with blanks (+ +).

We have made custom BizTalk schemas with the 6 custom EDIFACT record structures and map these to SQL datagrams. In the maps all fields are run through custom functoids to transpose empty/missing fields and missing sub records to a blank value (and transfer the name as column name) because the SQL Adapter can't import these as null by default.

My questions are:
1) Is there another simplified way to do the splitting with BizTalk 2006 or BizTalk 2006 R2 rather than building a customized Flat File Disassembler Pipeline?

2) Is there another way to import empty/missing fields and sub records into a SQL-database rather than using custom functoids.

Any help are appreciated.

Best regards,

Troels

[1839 byte] By [TroelsMNielsen] at [2008-1-9]
# 1
Hi Troels,
1). Yes. The 2006 R2 with brand new EDI subsystem take care of it. There are a huge set of EDI schemas, sure your schemas are inside this set.
EDI subsystem will debatch your EDIFACT to the EDI documents, etc.
2) not sure I completely understand the question. Usually I pass the Xml to SQL as is, without struggling with empty/null nodes. Let's SQL take care of it. SQLXML (OPENXML) transfer the missed nodes to Null values, and I work with simple UDF to transfer Empty-->null, null-->Empty, 0-->Null etc.
Regards,
Leonid Ganeline
http://geekswithblogs.net/leonidganeline/
LeonidGaneline-MVP at 2007-10-2 > top of Msdn Tech,BizTalk Server,BizTalk EDI and AS2...
# 2

Hi Leonid,

Thank you for your quick response.

1) I'm pretty sure that our Telekurs EDI-schema isn't inside the standard sets of EDI schemas in BizTalk 2006 R2.
It uses the Notation symbols and the message/segment/composite- and single-data element design as well as the 're-usability' of data elements, segments etc. as defined by the UN/EDIFACT standard. However, all field, segment and message designs are Telekurs-proprietary, i.e. they have not re-used the elements from the original UN/EDIFACT field dictionary. And twice a year the structure are changed (elements are added). Maybe standard EDI header, trailer and envelope information are missing too.

Is there a standard EDI Flat File Disassembler or other tools in BizTalk 2006 R2 that can help splitting and handling the 6 custom EDI-schemas we are using? (As far as I am aware the standard BizTalk 2006 Flat File Disassembler can't handle this).

2) As far as I am aware the SQL Adapter in BizTalk 2002 couldn't handle missing fields/elements by default when using SQL datagrams. I am not sure if I understand your solution. Are you using SQL datagrams or stored procedures? Is UDF a User Defined Function = Functoid.

Best regards,
Troels

TroelsMNielsen at 2007-10-2 > top of Msdn Tech,BizTalk Server,BizTalk EDI and AS2...
# 3
1) I assume you can use standard FF disassembler with your custom made schemas. And preprocessing could help with most of issues. (not sure about creating the custom schemas for EDI R2 Subsystem. The format of schemas is closed But very predictable.) Summary: custom preprocessor + FF disassembler.
2) I meant using the stored procedures, not updategramms, to save data to SQL. Yes, UDF is User Defined Function, sorry to abbreviation.
Regards,
Leonid Ganeline
LeonidGaneline-MVP at 2007-10-2 > top of Msdn Tech,BizTalk Server,BizTalk EDI and AS2...