• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Need help in designing a TIS Java project

#1 2009-07-31 16:04:59

achojar
New member
Registered: 2009-01-28
Posts: 8

Need help in designing a TIS Java project

We have multiple text files that have records in various formats as below, all fixed length without any delimiters:-

      BSP00000001…
      BSP00000002…
      …
      BSP00000099…

There are multiple formats in each file, but all lines have the same total length (136).

Each different record format (BSP..1 thru BSP..99) needs to inserted into different Postgres database tables. Is there a way in TIS to iteratively read records from files, then based on values of the first 3+8=11 positions, insert the rest of the fields into different tables? Note that the rest of the fields are of different lengths for each format.

For example, all BSP00000001… lines of files need to go into table:

bsp01_file_header_record
(
  standard_message_identifier    VARCHAR(3),
  sequence_number                   NUMERIC(8,0),
  standard_numeric_qualifier      NUMERIC(2,0),
  bsp_identifier                          VARCHAR(3),
  ticketing_airline_code_number  VARCHAR(5),
  handbook_revision_number      VARCHAR(3),
  test_production_status              VARCHAR(4),
  processing_date                       DATE,
  processing_time                       TIME WITHOUT TIME ZONE,
  iso_country_code                     VARCHAR(2),
  file_sequence_number             NUMERIC(6,0),
  reserved_space                       VARCHAR(90)
)

All BSP00000002… lines of files need to go into table:

bsp02_billing_analysis_cycle_header_record
(
  standard_message_identifier VARCHAR(3),
  sequence_number                NUMERIC(8,0),
  standard_numeric_qualifier   NUMERIC(2,0),
  processing_date_identifier    VARCHAR(3),
  processing_cycle_identifier   NUMERIC(1,0),
  billing_analysis_end_date     DATE,
  dynamic_run_identifier         CHAR(1),
  hot_reporting_end_date       DATE,
  reserved_space                  TEXT
)

and so on.

I would greatly appreciate if you could help us out. TIS version is:

"
Talend Integration Suite - Enterprise Edition

Version: 3.0.4
Build id: r22547-20090316-1930
"

Thanks!
atul

Offline

#2 2009-07-31 22:02:47

Volker Brehm
Member
Registered: 2007-04-03
Posts: 1139
Website

Re: Need help in designing a TIS Java project

Hi atul,

you could read your data with tFileInputPositional and three columns (3,8,rest). Then use tMap to split your data into different output streams depending on the first two fields. Each stream will get its own tExtractDelimitedFields, output component and whatever is needed to process the data.

In 3.1 new multi schema components are added. For example "tFileInputMSPositional" but I never used them until now.

Bye
Volker

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Need help in designing a TIS Java project

Board footer

Powered by FluxBB