You are not logged in.

Unanswered posts



Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at https://community.talend.com/. We look forward to hearing from you there!



#1 2014-09-10 09:00:21

VictorOkulov
Member
5 posts

VictorOkulov said:

OPC UA Component

Hi
I need to analyze the data from the Industrial Control System (PLC, SCADA)
These systems have OPC UA interface.
Amount of data in the Industrial Control System - Big Data

Offline

#2 2014-09-10 09:33:26

sanvaibhav
Member
1719 posts

sanvaibhav said:

Re: OPC UA Component

Hi Victor,

How you are planning to move data from your hardware systems to file system? If you can get data into the file system, then Talend could help to process them as required. Just few questions out of curiosity..
What is the Volume and Velocity of your Big Data? 
Do you want to process the data in real time? 
What is the frequency by which you will be offloading data from hardware to file system? 
What you want to do with that data?

Thanks
Vaibhav


Talend Certified Consultant

Offline

#3 2014-09-10 09:51:43

VictorOkulov
Member
5 posts

VictorOkulov said:

Re: OPC UA Component

There are 16 computers, each generates about 50 megabyte per day.
Offloading data plan in the database. Tell me, what a database is best suited for BigData?
We want to process the data in real time.
Based on the data necessary build graphs

Offline

#4 2014-09-11 10:13:35

VictorOkulov
Member
5 posts

VictorOkulov said:

Re: OPC UA Component

subject - https://www.mapr.com/blog/ultimate-3-mi … d-opentsdb

Last edited by VictorOkulov (2014-09-11 10:14:09)

Offline

#5 2014-09-11 21:36:41

John Gillerman
Guest

John Gillerman said:

Re: OPC UA Component

Victor,
Putting the OPC UA data into a database is not a good idea.  Instead, write an OPC UA client that converts UA calls to XML messages that you can access from Talend applications.  I am very interested in this work and can help you if you want. 
John Gillerman

#6 2014-09-12 03:35:09

VictorOkulov
Member
5 posts

VictorOkulov said:

Re: OPC UA Component

I tried to configure the OPC UA REST server - http://node-opcua.github.io/
But unfortunately was not able to set it up - https://github.com/node-opcua/node-opcua/issues/10
John, I would be very grateful if you could help me

Last edited by VictorOkulov (2014-09-12 03:35:54)

Offline

#7 2014-09-12 07:30:58

sanvaibhav
Member
1719 posts

sanvaibhav said:

Re: OPC UA Component

Hi,
There are 16 computers, each generates about 50 megabyte per day.
>> Only 50MB per day is not a Big Data... Data could be much bigger... GB/TB/PB of data with very high velocity (per minute/hour/day) could come in the perview of Big Data
Offloading data plan in the database. Tell me, what a database is best suited for BigData?
>> depends upon what and how you want to play with the data
We want to process the data in real time.
>> Real time process of data is not in the scope of ETL, it is in the scope of Dashboard development. There are many vendors who work on real time data display. Concept is dashboard queries database in realtime every second and changes data which is new. In order to support real time data display, what is the frequency of data offloading to database from your systems. This can't be real time as you are off loading with particular frequency.

Based on the data necessary build graphs
This is again in the scope of Report and Dashboard development. Few graph components are available which can draw graphs for you, but that is limited to line and bar charts etc.


I hope you got an idea. Let me know if you have any further questions...


Thanks
Vaibhav

Last edited by sanvaibhav (2014-09-12 07:40:01)


Talend Certified Consultant

Offline

#8 2014-09-12 20:19:25

John Gillerman
Guest

John Gillerman said:

Re: OPC UA Component

Victor,
I believe that bring UA data on to a Talend integration bus is a very valid use case.  Importantly, Talend supports the notion of information models which is also the case of UA of course.  IMHO, it makes sense to write a UA client that is used to set up UA subscriptions to the backend UA server.  Once the UA client starts receiving UA DataChange events, it just publishes them on to the Talend Bus.  Talend Applications can then subscribe to Talend messages on the Talend Bus.  The limitation of this of course is that the monitored item group is set up in the UA client and shared by all Talend message consumers.  You could set up multiple subscriptions in the client and publish them using different message ids on the Talend bus.  But this requires coordination of configuration between the Talend applications and the UA client.  I recommend that you start with this.  Over the longer term, you can also get CreateMonitoredItems and Subscribe messages on to the Talend bus, I would not start out doing this.
John

#9 2014-09-12 20:24:08

John Gillerman
Guest

John Gillerman said:

Re: OPC UA Component

With regard to using a database and rest, this adds a very large layer between UA and Talend message consumers.  The database should either be a UA client (preferred) or a Talend message subscriber.  Feel free to contact me directly at john@gridcloudsys.com.

#10 2014-09-23 07:16:52

VictorOkulov
Member
5 posts

VictorOkulov said:

Re: OPC UA Component

OPC UA Component does not make sense?

Offline

#11 2014-11-22 07:39:36

john gillerman
Guest

john gillerman said:

Re: OPC UA Component

why not? IMHO it makes a lot of sense.

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy