X12 Character Set Differences
Hi,
In the EDI pipeline components and in the EDI party configurations there is a X12 Character setting with the values of Basic,Extended, and UTF-8/Unicode.
What are the differences between the 3. I couldn't find it in the documentation.
Thanks,
Mike
Basic allows the following
A...Z
0...9
! “ & ’ ( ) * + , - . / : ; ? = “ ” (space)
Extended allows the following
All of Basic
a..z
% @ [ ] _ { } \ | < > ~ # $
UTF8/Unicode allows every thing including Basic and Extended character set. So if you chose Basic in party configuration, and have a lower case character in the payload, that document will be suspended since Basic does not allow lower case. That payload will pass validation with Extended.
Hope this helps
Mohsin
Mohsin,
I am using Biztalk 2006 R2 Beta release and when I look at the properties for the EDIReceive pipeline component, the CharacterSet is set to UTF8. According to your earlier comments, it appears that UTF8 should allow the special character >. However when I test it with a sample EDI-820 file that has > it does not work. Taking out that special character > makes it work. I have even tried changing the EdiDataValidation property for the receive pipeline to False and that did not work either. Let me know if there is anything else I can try to resolve this issue.
Are you disassembling the file (flatfile to XML on the receive side) or serializing (XML to flatfile on the send side)? If its on the send side, you need to escape this character since it is a special character in XML. Also can you post your instance so that I can take a look at it?
Thanks
Mohsin
I am disassembling the file and using the standard X12_00304_820.xsd schema. Attached below is the error message I receive
Error encountered during parsing. The X12 transaction set with id '0001' contained in functional group with id '176994', in interchange with id '000178113', with sender id 'B215T ', receiver id '005037726 ' is being suspended with following errors:
Error: 1 (Field level error)
SegmentID: NTE
Position in TS: 66
Data Element ID: NTE02
Position in Segment: 2
Data Value:
6: Invalid character in data element
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
The sample 820 file that is causing the above error is pasted below. As you can see the special character > is used in the NTE segment of this file.
ISA~00~ ~00~ ~ZZ~B215T ~01~005037726 ~070907~2008~U~00401~000178113~0~P~>*
GS~RA~B215T~005037726~070907~2008~176994~X~003040*
ST~820~0001*
BPR~I~+1689817.11~C~ACH~~01~700 00000~~700~9B69KA0000~~~~~~070910~VEN*
TRN~3~000312071~9B69KA0000*
N1~PR~FORD MOTOR COMPANY~92~B69KA*
N2~CENTRAL ACCOUNTING SERVICES*
N1~PE~FLUID HANDLING SYSTEMS~92~B215T*
PER~AP~~TE~408-856-3322*
ENT~1~~~~SU~92~B215T*
N1~SU~FLUID HANDLING SYSTEMS*
RMR~IV~ G061478-00~~-749.00~-749.00~+.00*
NTE~PAY~ FOR DEDUCTIONS THIS MONTH REFER TO>*
NTE~PAY~ CONCERN NO.> G061478 QR NO.> G061478-00*
NTE~PAY~ CALL> DOROTHY MARTIN 816-459-1246*
NTE~PAY~ FOR DEDUCTIONS THIS MONTH REFER TO>*
NTE~PAY~ CONCERN NO.> G061478 QR NO.> G061478-00*
NTE~PAY~ CALL> DOROTHY MARTIN 816-459-1246*
REF~TN~7248C0010386B*
REF~PE~AP06*
RMR~IV~ G061518-00~~-509.76~-509.76~+.00*
NTE~PAY~ FOR DEDUCTIONS THIS MONTH REFER TO>*
NTE~PAY~ CONCERN NO.> G061518 QR NO.> G061518-00*
NTE~PAY~ CALL> MICHAEL J. TUCKER 816-414-5756*
NTE~PAY~ FOR DEDUCTIONS THIS MONTH REFER TO>*
NTE~PAY~ CONCERN NO.> G061518 QR NO.> G061518-00*
NTE~PAY~ CALL> MICHAEL J. TUCKER 816-414-5756*
REF~TN~7248C0010387B*
REF~PE~AP06*
SE~28~0001*
GE~1~176994*
IEA~1~000178113*
This error is since you are using '>' in ISA level as a component seperator. Same seperator is coming in data filed also which is causing the problem..Try using a different character ( eg: ' ) and test it..
Genuine- Thanks for this tip. After I made the change to the ISA segment in the input file to replace the > with ' I was able to successfully process the file using Biztalk R2. The ISA segment as it came orignially was as below
ISA~00~ ~00~ ~ZZ~B215T ~01~005037726 ~070907~2008~U~00401~000178113~0~P~>*
and after modification is was as below
ISA~00~ ~00~ ~ZZ~B215T ~01~005037726 ~070907~2008~U~00401~000178113~0~P~'*
I have a concern regarding this though. In my example the component element seperator is ~ and the segment element seperator is *. Why should Biztalk care about the > in the ISA segment and not allow that character in the notes segment. Besides this is a file that is coming from Ford and I do not have any control over the use of any other character other than > in the ISA segment. Let me know your thoughts.
In your sample file, '>' is used as a component seperator*.
In your case I would recommend to use a pre processing pipeline component which would replace the '>' character with any other valid character..
*correction
Genuine- As you mentioned in your reply "In an X12 interchange, the segment terminator is defined as the character in the last character position of the ISA segment" and if you look at my sample file, the last character in the ISA segment is * and not >. In fact if you look at all the lines in the sample file, the last character of each line is an * followed by CR/LF which I would think denotes that * is the segment terminator. Am I missing something? Please let me know.
Yes, you are right.. Then the answer is simple, ISA16 which is the component seperator is being used as part of data in NTE02 filed.