You are not logged in.
Announcement
Unanswered posts
|
Pages: 1

Hi Everybody,
I want to do sth about FTP on TOS. I have a common subdirectory under the root directory on my FTP Server. There are a lot of XML files (same formaatted). The naming convention has a rule like ID-DDMMYYYY.xml I want to get this file and read each record in it. Also need the properties of file name. Then want to save all these info. to my Oracle DB Table. Can you help me about the roadmap that I should follow. For example which components and which order. TOS is too flexible that I lost myself inside the forum posts.
Best Regards,
Mevlut
Offline
Hello
First, use tFTPGET to get all the xml file into a folder, then use tFileList to iterate each file and extract the records from the file, merge all the records and insert them into DB. The job looks like:
tFtpGet
|
onsubjobok
|
tFileList---iterate---tFileProperties-main--->tJavaRow--oncomponentok-->tFileInputXML--main-tMap---main---tOracleOutput
tJavaRow: get the file properties and store them into global vars, eg: globalMap.put("basename",input_row.basename);
tMap: define some new columns for file properties, for exmaple, add a new column call basename and set the expression as: (String)globalMap.get("basename")
Best regards
Shong
Offline

Hi shong,
I will try and back to here with my results and feedbacks...
Thank you very much...
Mevlut
Offline

Hi again,
Q.1) tFileList? tFTPFileList?
Q.2) There is no "main" opiton from both tFileList & tFTPFileList. There is only iterate. So ?
Thanks...
Offline
Hello
Q.1) tFileList? tFTPFileList?
==>tFileList
Q.2) There is no "main" opiton from both tFileList & tFTPFileList. There is only iterate. So ?
==>Yes, it is not a input/output component, so thare is not a 'main' connector, please see the user documentaion and learn its usage.
Best regards
Shong
Offline

You said "tFileProperties-main--->tJavaRow" :-))))
Thank you very much for responses...
Mevlut.
Offline

Hi againn,
I tried what you said in order (as you suggeted) but havent succeded yet :-))
I have lots of xml files. I declared 4 context variables because of basename of my files have a format. I split them in cod pane of tJavaRow component and assign them to context variables.
Then I use the "((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))" expression where the file needed and it works. untill the tFileInputXML component.
My XML :
<?xml version="1.0" encoding="UTF-8"?>
<stats>
<stat>
<key>002</key>
<value>12032019</value>
</stat>
<stat>
<key>003</key>
<value>12032010</value>
</stat>
</stats>
And my output table (BKS_STAT) has columns (ID,KEY,VALUE)
Q.1) How can I use tFileInputXML component for the xml files that have same format but that will be apparent on run-time ?
Q.2) Also use tMap usage is not clear without knowing the answer of the Q.1 !!!
Q.2) Can I use Sequence (in Oracle) for ID column ?
Q.3) is sing Context for split basename into 4 pieces (I will use them to insert BKS_STAT table with the ID, KEY, VALUE) sensible ?
Thank you very muck for your responses,
Best regards,
Mevlut.
Last edited by mbecerikli (2010-03-15 16:39:16)
Offline

Can Anybody (cantoine, esabot, shong) Help Me ???
Last edited by mbecerikli (2010-03-17 10:56:20)
Offline

NO COMMENT ???
Offline
Hello
Sorry for delay!
Q.1) How can I use tFileInputXML component for the xml files that have same format but that will be apparent on run-time ?
As a newbie, it is better to create the xml metadata follows the wizards. In this case, the basic setting of tFileInputXml looks like as my screenshot.
Q.2) Can I use Sequence (in Oracle) for ID column ?
Of course, see [Forum, topic 1488] Oracle sequence while insert.
Q.3) is sing Context for split basename into 4 pieces (I will use them to insert BKS_STAT table with the ID, KEY, VALUE) sensible ?
Can you take an example to explain your request?
Best regards
Shong
Offline

I am sorry for persistant demand Because of being placed in difficult circumstances! But I persuaded my manager to use TOS after a long and hard demo. But I achieved that we took a full liecence to use TOS :-)
My Main Goal is to take data from our customers via XML. The put put their XML files,whose basenames are formatted (XXX-Cust_ID-YEAR-TERM-SYSDATE.xml) special to them , in a common FTP folder.
I will take these files from FTP and copy to my locale and then insert (second phase has also UPDATE but not now) datas into Oracle with both splitting basename and xml nodes together.
Now I follow the order you suggested to me : [ tFTPGet -->(iterate)--> tFileList -->(iterate)--> tFileProperties -->(main)--> tJavaRow ] -->(OnComponentOk)--> [ tFileInputXML -->(Main)--> tMap -->(Main)--> tOracleOuptput ]
I got the files to my locale folder. Get the property of each file and parse the info by splitting basename into Context variables in tJavaRow. But at this point ı stopped. My files not predefined, they will be live at run-time. So I have to set properties of tFileInputXML & tMap manually.
For tFileInputXML;,
--------------------------
filename => "((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))"
LoopXPathQuery => "/stats/stat"
Schema => ???
Mapping => ???
--------------------------
And My Xml structure:
--------------------------
<?xml version="1.0" encoding="UTF-8"?>
<stats>
<stat>
<key>002</key>
<value>12032019</value>
</stat>
<stat>
<key>003</key>
<value>12032010</value>
</stat>
</stats>
--------------------------
What should I do about the tFileInputXML (firstly) properties?
Thanks for responses again,
Mevlut.
Offline

Shong,
Now I' ve seen your Screenshot. I m trying now...
Thanks...
Offline
Hello
What should I do about the tFileInputXML (firstly) properties?
Did you see my screenshot? I have showed how to define the schema, mapping based on your xml file.
As I said, you have to learn how to create xml metadata and how to use the metadata on tFileInputXML component, just create a simple job to learn how to extract records from a xml file and output it to console. for example:
1)Go to Repository-->Metadata and right click on File xml, select 'create file xml' to create a xml metadata follows the wizards.
2)Create a simple job,
tFileInputXML--main--tLogRow
on tFileInputXML, choose 'repository' on property type list and choose the xml metadata just created by you.
Best regards
Shong
Offline

Shong,
Probably you, first wrote than attached the screenshot so I'd written my message before I saw your screenshot.
I did what you'd said and it worked. I m really appreciated for the help. Thank you very much.
I' m stronger with my TOS from now on! :-))))
Now It' s time to improve the functionality of my job! New task is the control of the element values of xml file;
1-) Check Whether the value null
2-) Check the existence of values in the dimension tables (For instance; my output table is "orders" and I will check the the existence of the "customer_id" column from "customer" table)
I will collect the all results and then send mail them to a mail address. And then die and rollback the transaction for that file(!)
If it is sensible, How can I achieve this (Maybe tLogCatcher, tAssert.....etc.) ?
Thanks again !!!
Best regards
Mevlut
Last edited by mbecerikli (2010-03-18 13:12:01)
Offline
Hello
1-) Check Whether the value null
Yes, you can check the value if it is null and handle the null value on tMap, for example:
tFileInputXML--row1-->tMap--tMysqlOutput
row1.columnName==null?value1:value2
There are many such case on forum, pleasr search and learn it.
2-) Check the existence of values in the dimension tables (For instance; my output table is "orders" and I will check the the existence of the "customer_id" column from "customer" table)
Select the 'insert or update' item on the action on data list of txxxOutput component. Maybe you need do some inner join between order table and customer table.
I will collect the all results and then send mail them to a mail address. And then die and rollback the transaction for that file(!)
If it is sensible, How can I achieve this (Maybe tLogCatcher, tAssert.....etc.) ?
Please see the user documentaion and learn the following component:
tMysqlConnection, tMysqlCommite, tDie, tSendeMail.
Best regards
Shong
Offline

OK! I got how to use expression editor (java) and use join operations. And with tDie, I stop the job which is running. And send mail with tSendMail.
But I could not get how to collect error messages which I got from tMap (row1.columnName==null?value1:value2). I can store them in a context variable.
But How can I fire to put them in the expression editor in tMap. Checking with this and Join Operations, I just filters the data (?).
Be sure I am reading the Reference and User Guides and searching your on the internet. But I am new to TOS and trying to improve as fast as I can
I'm sorry by taking your valuable time :-(
Thanks,
Mevlut.
Offline
Pages: 1