====== XML and Revelation: A White Paper (General) ======
====== ======
==== Created at 03 DEC 2002 03:07PM ====
Mike Ruane and Sean FitzSimons, Revelation Software
**XML and Revelation**
****
This white paper is intended as an introduction to XML for Revelation Software developers and users. It will inform the reader of:
What is XML
Why Use XML?
Terms andDefinitions
Why would I useXML
Using XML inHow does one I use XML in OpenInsight
XML Tools in an OpenInsight
**What is XML?**
****
XML Stands for Extensible e Markup Language. It is based upon an earlier markup language called SGML, or Standardized General Markup Language. A markup language is a special set of indicators, called tags,,that that indicate how information should be interpreted. XML is designed to describe data. XML uses a DTD or Schema to describe data. An XML file is a Document, made up of Tags that describe elements, that may or may not have Attributes.
**Why Use XML?**
****
First, a few facts about XML. It is not a programming language. It is a markup language only, used to describe data. In order to manipulate XML one needs to use a programming language, such as Basic+. Also, XML will not replace HTML. HTML is used to mark up documents for display purposes. XML is used to markup documents for data purposes only.
Why should Revelation developers and users be concerned about XML? There are a number of reasons.
Structured text Format.
Designed with the Internet in mind: As more and more business is run across the internet, and as more information exchange occurs, XML will probably be the format of choice, instead of CSV files, EDI layouts and the like.
Processing technology is widespread and cheap.
Human Readable: If you've ever looked at a flat file, but you don't know the file layout, it is very difficult to determine where one field ends, and where the next begins. It s also difficult to determine which field is used for what purpose. That s not true with XML.
Schema Agreement allows exchangeable documents.
Enables E-business.
Easily Internationalized.
Easy for MV Developers Hierarchical
XML can represent our Multivalued data very well.
XML and MV: Both delimiter-based.
XML Schema and MV Dictionary Both Metadata.
XML embraces Multivalued Data.
MV syntax ideal for manipulating XML.
Industry is spending billions to be in the delimited solution space and we are there now.
**XML and Revelation Developers**
****
XML is easy for Revelation developers. XML is self-describing data. It is a series of definitions of data, and then the data - sound familiar? It s the same model used by Revelation Software since its inception over 20 years ago. XML will not replace databases. In fact, there are products made specifically to be databases for XML. Revelation is an excellent database for XML.
**Terms and Key Definitions**
****
** **There are a number of key terms that must be understood when working with XML.
Element
Attributes
Document
NameSpaces
Schema
DTD
SAX
DOM
**Element**
An element is the basic unit in an XML document. It contains a Start Tag, an End tag, and data between the two.
E.g.
data
Tag Names are Case Sensitive
E.g. does not equal does not equal
Tags can be empty
E.g. instead of
**Attributes**
An attribute is a name-value pair separated by an equal sign = . Attributes are optional; they don t need to be a part of an XML file. Attributes can accept default values; Elements cannot.
**Document**
A Document is an XML element that may have nested XML elements. It must be well formed. By well formed, we mean it must always have a matching closing tag, and it must be completely nested. So
...... works and is well-formed, but the following is not
The image below shows an example of a well-formed XML document.
{{kb0222_1.png}}
The next image,however shows XML that is not well-formed. Note that the final CD and YEAR tags are out of order.
{{kb0222_2.png}}
When you look at this document in Internet Explorer, the following error appears:
{{kb0222_3.png}}
**Namespace**
A Namespace is a collection of names that can be used as Element or Attribute names in an XML document. If The Namespace is identified by a Uniform Resource Identifier (URI). A URI can be either a Uniform Resource Locator (URL) or Uniform Resource Number (URN). A URL can be real or made up - it does not really have to exist. Finally, a namespace can be declared explicitly or by Default.
**XML Example**
****
{{kb0222_4.png}}
**Schema**
A Schema is an XML-based syntax fordefining how an XML document is marked up. This is an external file; that is, it is not located within the XML file. A schema allows the use use of Namespaces. Schemas support datatypes, and can define complex structures. Complex structures
**Key Definitions: Schema Example**
****
{{kb0222_5.png}}
****
**DTD**
DTD stands for Document Type Definition. A DTD validates XML and SGML Documents. DTDs do not follow XML rules - they don t support Data Types, and they can t specify complex relationships. In general, DTD is older technology that is falling out of use. OpenInsight s XML implementation does not use or support DTDs.
**DTD Example**
**SAX**
SAX stands for Simple API for XML. It is used by event-based applications. Its primarily a Java interface. With SAX there is no need to load whole XML file into memory. However, SAX is read only!
**DOM**
DOM stands for Document Object Module. This is a process that converts an XML document into a collection of Objects. DOM treats a document as a Hierarchical structure. DOM is also known as Random Access method.
DOM is the method that is used by OpenInsight.
**Why Use XML**
Why should Revelation developers and users be concerned about XML? There are a number of reasons.
Structured textFormat
Designed with the Internet in mind. As more and more business is run across the internet, and as more information exchange occurs, XML will probably be the format of choice,instead of CSV files, EDI layouts and the like.
Processing technology is widespread and cheap.
Human Readable: If you've ever looked at a flat file, but you don t know the file layout, it is very difficult to determine where one field ends, and where the next begins.It s also difficult to determine which field is used for what purpose. That s not true with XML.
Schema Agreementallows exchangeable documents
Enables E-business
EasilyInternationalized
Easy for MV Developers Hierarchical
XML can represent our Multivalued data very well.
Right Time
Delimited Data
XML and MV: Both delimiter-based
XML Schema and MV Dictionary Both Metadata
XML embraces Multivalued Data
XML is replacing existing EDI systems
MV syntax ideal for manipulating XML
Industry is spending billions to be in the delimited solution space.
We are there now
**XML and Revelation Developers**
XML is easy for Revelation developers. XML is self-describing data. It is a series of definitions of data, and then the data - sound familiar? It s the same model used by Revelation Software since its inception. XML will not replace Databases. In fact, there are products made specifically to be databases for XML. Revelation is an excellent database for XML.
But First, a few facts about XML. It is not a programming language. It is a markup language only, used to describe data. In order to manipulate XML one needs to use a programming language, such as Basic+. Also, XML will not replace HTML. HTML is used to mark up documents for display purposes. XML is used to markup documents for data purposes only.
**XML Application Design: DOM**
****
The flowchart below shows how DOM is implemented in OpenInsight. An XML file is read into memory and parsed into logical records. These records are then stored either in memory, or written to an OpenInsight file. Data from either this file, or other, existing OpenInsight files can then be written back out to an XML file.
{{kb0222_6.png}}
**Using XML in OpenInsight**
****
OpenInsight developers can create their own XML routines - it s fairly easy to do. Revelation Software has also provided a number of routines to assist in the importing and exporting of XML data. All of the programs documented below are provided, with source code, as of OpenInsight version 4.1.
**XML_ _Importer**
This routine, as the name suggests, is used to import XML data into OpenInsight files. It takes the following parameters: Instead of creating a template every time you want to import XML data into OpenInsight, you can call the XML_IMPORTER routine directly. It is a function, and has the following syntax and parameters:
//XML_Importer////(////parent, XML_File, Batch_Flag, Schema_Name, key_tag, target_tags, Dict_Mappings, OI_Data_File, Create_Flag, Template_Name, Storage_Opts)//
////
|**Parameter** |****\\ \\ |**Description** |
|**Parent** |****\\ \\ |The parent window of the calling process. |
|**XML_FILE** |****\\ \\ |The name of the operating system file you will be importing from. |
|**Batch_Flag****** |****\\ \\ |A Boolean value indicating whether or not the program is being run in batch mode. In batch mode, the program will not display informational messages. Pass a 1 to indicate batch mode, and a 0 or null for interactive mode.|
|**Schema_Name****** |****\\ \\ |The name of the Schema file associated with the XML file. |
|**Key_Tag****** |****\\ \\ |The tag in the XML file that will be used as a key in the OpenInsight file. |
|**Target_Tags****** |****\\ \\ |A value-mark delimited list of tags that will be extracted and put into the OpenInsight file. |
|**Dict_Mappings******|****\\ \\ |A value-mark delimited list of dictionary field names that the target tags will be mapped into. Based upon Storage_Opts, this parameter may be null. |
|**OI_Data_File****** |****\\ \\ |The name of the OpenInsight file into which the XML data will be placed. |
|**Create_Flag****** |****\\ \\ |A Boolean flag indicating whether or not the OI_Data_File should be created. Note: This functionality will not work in Runtime versions of OpenInsight. |
|**Template_Name******|****\\ \\ |The name of an import template that has been saved. If you want to run imports programmatically, provide this parameter, otherwise leave it null. |
|**Storage_Opts****** |****\\ \\ |Indicates how data will be stored in fields in the OpenInsight file. It can have one of three possible values. |
|\\ \\ |C |Create New F type fields. Note: Cannot be implemented in Runtime versions of OpenInsight. |
|\\ \\ |E |Map to Existing F type fields. |
|\\ \\ |D |Create a new F type field with embedded with raw XML; also create new Symbolic fields to extract the XML data. Note: Cannot be implemented in Runtime versions of OpenInsight. |
**Extract_XML_Schema_Name******
This program will extract the name of an XML file s schema when passed the XML file.It is a function, and has the following syntax and parameters:
//Extract_XML_Schema_Name////(////XML_File////, XML_Rec)It takes the following parameters://
////
|**Parameter** |**Description** |
|**XML_File******|The XML file path and name. |
|**XML_Rec****** |The XML file that has been read into memory.|
XML_File
The XML file path and name.
XML_Rec
The XML file that has been read into memory.
**Get_XML_Value******
This function is used to extract tag values from records that have raw XML data stored in them. It is a function, and has the following syntax and parameters:
//Get_XML_Value////(////Target_Tag////, XML_Rec)//
////
|**Parameter** |**Description** |
|**Target_Tag******|The tags within the XML from which you wish to extract values.|
|**XML_Rec****** |The XML record containing the data you wish to extract. |
**Inet_OI_XML******
****
This program will create XML and XML Schema documents. It is a function, and has the following syntax and parameters:
//Function Inet_OI_Xml(Request, Cmd, Dosfile,Select_Stmt, Template_Name)//
////
|**Parameter** |**Description** |
|**Request** |The CGI parameter passed when called through the Internet. If this program is not called via the Internet, this parameter should be null. |
|**Cmd****** |A List statement containing the OpenInsight table and field names to be extracted from the database and published as XML. |
|**Dosfile****** |The full path and filename of the XML file to be published. |
|**Select_Stmt****** |Optional. An RLIST select statement may be used to select subsets of the data to be extracted. |
|**Template_Name******|The name of an export (XML Publisher) template that has been saved. If you want to run exports programmatically, provide this parameter, otherwise leave it null.|
**Create_XML_XSD_Schema******
****
This program will create will create XML Schema definition documents. It is a function, and has the following syntax and parameters:
//Create_XML_XSD_Schema////(////Tablename////, Rpt_Fields, Dos_Path, DosFile)//
////
|**Parameter** |**Description** |
|**Tablename****** |OpenInsight table to be published as XML. |
|**Rpt_Fields******|A value mark delimited list of dictionary field names to be included in the published document.|
|**Dos_Path****** |Unassigned. Pass as a null parameter. |
|**DosFile****** |The full path including filename of the XML file for which the XSD is being created. |
**XML Tools in OpenInsight**
****
As of OpenInsight version 4.1, a new XML Workspace was introduced into the toolset. The XML workspace consists of 5 tools: the XMLImporter, the XML Publisher, the XML Explorer, the XPATH window, and the XML Configuration window. These tools are well documented in the OpenInsight Online Documentation, so we ll just describe them briefly here.
**__XML Importer__**
The XML Importer is a front-end screen to the XML_IMPORTER subroutine. It allows the user to create, run, and save import templates. This tool is only available in Development environments; it will not work in runtime systems.
The XML Importer is a front-end to the XML_IMPORTER subroutine. It is used to create templates that are used to import XML data into OpenInsight files. When an XML import template is created, it can either be run, saved to file, or both.
Please note that this tool is only available on development copies of OpenInsight; it will not work on Runtime systems.
**The XML Importer Window:**
****
{{kb0222_7.png}}
****
****
****
****
|**Control** |**Description** |
|**Filename** |The OS-based XML filename. Clicking on the Options button next to the edit line will display a dialog box allowing the user to choose an Operating System filename. |
|**Schema** |The name of the schema associated with the XML file that was entered in the Filename field. Clicking on the Options button next to the edit line will display a dialog box allowing the user to choose an Operating System filename. |
|**Tag to Use as Key**|The name of the tag in the XML file that will be used as a key in the OpenInsight file. If there is no tag in the XML file that can be used as the key, the user can choose Sequential Key Counter from the drop down list. |
|**OI Filename** |The OpenInsight file into which the XML data will be stored. Clicking on the Options button next to the edit line will display a popup of existing OpenInsight tables available. |
|**Create New File** |When entering a filename that does not already exist, click on this checkbox. Doing so will create a new table in the current OpenInsight database being worked on and attach the file. |
|**Target Tags** |Edit table to determine which XML tags to import into the OpenInsight file. If the OpenInsight file is being created the Tag Name will be the only column displayed. This is because if it is a new table, there are no dictionary items to map to. |
|**Data Storage** |Indicates how the XML data will be stored in the OpenInsight files. The choices are:\\ \\ store raw XML in the first available field position, and create symbolic fields to display the data;\\ \\ store the data in existing fields that have been mapped;\\ \\ store the data in newly created F type fields.|
**__XML Publisher__**
The XML Publisher is a front-end screen to the INET_OI_XML subroutine.. It allows users to create, run, and save XML Export templates. This tool is only available in Development environments; it will notwork in runtime systems.The XML Publisher is a front-end screen to the INET_OI_XML subroutine. It is used to create templates that are used to export data from OpenInsight files to XML files. When an XML Publisher template is created, it may be run, saved to file, or both.
Please note that this tool is only available on development copies of OpenInsight; it will not appear in Runtime systems.
**The XML Publisher Window:**
****
{{kb0222_8.png}}
****
|**Control** |**Description** |
|**Tablename****** |The name of the OpenInsight file whose data will be exported. |
|**Available Fields**|A list of all fields as defined in the dictionary for the table in the Tablename field. |
|**Selected Fields** |A list of those fields, which will be exported to the XML File. |
|**Select Statement**|A select statement that will be used to select a subset of the file to be exported. This field is optional.|
|**Build Statement** |Clicking this button will call a window that will allow the user to create select statements. |
|**Check Syntax** |** **Clicking this button will check the syntax of the Select statement, if there is one. |
|**OS Filename** |The DOS path and filename that will be created. |
|**Generate XML** |Clicking this button will generate the XML file. |
**__XML Explorer__**
The XML Explorer is a window that allows users to see a hierarchical representation of an XML file.This tool is used to view the hierarchical XML structure of an XML file from within an OI screen. Choosing XML Explorer from the Tools menu executes the XML Explorer window.
{{kb0222_9.png}}
Clicking on the Choose File button will call up the Open File dialog box, listing XML files. Choose a file by clicking on it and clicking on the OK button. The filename will then be put into the OS Filename field. Clicking on the Load button will cause the explorer window to appear.
{{kb0222_10.png}}
In hierarchical structures of XML, clicking on the plus ( + ) or minus ( - ) keys will make the listing expand or contract.
**__XPath__****__ Window__**
The XPath window is provided to allow users to enter XPath statements against an XML file and see the results.
The XPath window is provided to allow users to enter XPath statements and view the results. XPath statements are usually passed into an application via web requests. This window is provided to let users and developers experiment with and evaluate XPath commands.
Choosing XPath from the Tools menu executes the XPath window.
{{kb0222_11.png}}
|**Control** |**Description** |
|**OS Filename** |The name of the XML file that will be used. Clicking on the **Choose File** button will bring up the Open File dialog box. |
|**Load** |Clicking on this button will cause the XML file to appear for viewing. |
|**XPath**** Statement**|An editline for entering **** XPath commands. |
|**Execute** |Clicking on this button will run the command within the XPath Statement. |
|**Results** |Display the results of the XPath statement. |
**__XML Configuration Window__**
The XML Configuration window is used to set up OpenInsight specific settings for dealing with XML files, including matching up of Datatypes and reserved words.
The XML configuration screen is used to enter and/or modify configuration settings for OpenInsights XML functionality. It contains the datatype mapping that is used by OpenInsight when creating Dictionary items for new files.
Choosing XML Configuration from the Tools menu executes the XML Configuration window.
{{kb0222_12.png}}
|**Control** |**Description** |
|**XML Datatypes** |A list of all allowable XML datatypes. |
|**OI Datatypes** |The equivalent datatype for the XML datatypes. When no specific mapping is used, the system defaults to VARCHAR.|
|**Keywords** |A list of those words which may be imported or exported by an XML process, but may be reserved words. |
|**Keyword Prefix**|The character that is added to the beginning of the keywords. |
**XML Resources**
****
The following websites are very useful in gaining an understanding of XML:
__www.Biztalk.org__ - Sitefrom Microsoft that offers schemas from many industries
__www.w3schools.com/xml__ - manysources, tutorials
__www.xml101.com__ Learning site
__www.xml.org__ Industry Portal
__www.zvon.org__ - The Guide to the XML Galaxy tutorials, references.