12 月 3 日、4日，2022 Apache IoTDB 物聯(lián)網(wǎng)生態(tài)大會(huì)在線上圓滿落幕。大會(huì)上發(fā)布 Apache IoTDB 的分布式 1.0 版本，并分享 Apache IoTDB 實(shí)現(xiàn)的數(shù)據(jù)管理技術(shù)與物聯(lián)網(wǎng)場(chǎng)景實(shí)踐案例，深入探討了 Apache IoTDB 與物聯(lián)網(wǎng)企業(yè)如何共建活躍生態(tài)，企業(yè)如何與開(kāi)源社區(qū)緊密配合，實(shí)現(xiàn)共贏。
我們邀請(qǐng)到德國(guó)普戈曼公司 CEO，兼天謀科技?xì)W洲技術(shù)負(fù)責(zé)人 Dr. Julian Feinauer 參加此次大會(huì)，并做主題演講——《Apache IoTDB 在德國(guó)汽車(chē)生產(chǎn)線多級(jí)數(shù)據(jù)同步中的應(yīng)用實(shí)踐》。以下為英文內(nèi)容和中文翻譯全文。
Hello everybody, my name is Julian Feinauer. I'm speaking to you from Germany. I'm working for a company called Pragmatic Industries, but I’m also a member of the Apache IoTDB PMC.
I can report you today about a very interesting application with Apache IoTDB that we have done together with a German automotive OEM. We used one of its rather unique features, namely the multi-master syncing ability, to generate a multi-level data syncing application to handle quite large amount of data that come up in automotive production.
Here's a short agenda what I want to talk about. I would shortly give a motivation introduction into the background of the project because the data was generated from electric motors, which are used to supplement pneumatic clamps, which are used a lot not only in automotive industry, but also in many other industries.
And the main part will be about the data collection and architecture we applied using Apache IoTDB, and then I'll end with a short summary of what we've done and what we've learned.
大家好，我是 Julian Feinauer。我來(lái)自德國(guó)，在一家名為 Pragmatic Industries （普戈曼）的公司工作，同時(shí)也是 Apache IoTDB PMC 的成員。
今天我將為大家分享一個(gè)非常有趣的 Apache IoTDB 應(yīng)用案例，一個(gè)我們與某德國(guó)汽車(chē)生產(chǎn)商共同完成的項(xiàng)目。我們使用了 IoTDB 的一個(gè)相當(dāng)獨(dú)特的功能，即多主機(jī)同步能力，來(lái)完成一個(gè)多級(jí)數(shù)據(jù)同步應(yīng)用程序以處理汽車(chē)生產(chǎn)過(guò)程中的海量數(shù)據(jù)。
這里是我將要做的報(bào)告目錄。我將簡(jiǎn)要介紹一下項(xiàng)目的背景，數(shù)據(jù)來(lái)源于電動(dòng)機(jī)，作為氣動(dòng)夾緊器的改進(jìn)。氣動(dòng)夾緊器不僅被大量應(yīng)用于汽車(chē)行業(yè)，在很多其他行業(yè)也被廣泛使用。
報(bào)告的主要內(nèi)容將介紹我們使用 Apache IoTDB 實(shí)現(xiàn)的數(shù)據(jù)收集以及設(shè)計(jì)架構(gòu)，然后我將對(duì)我們實(shí)現(xiàn)的增益與學(xué)到的經(jīng)驗(yàn)做簡(jiǎn)短的總結(jié)。
01 Motivation 項(xiàng)目動(dòng)機(jī)
So let's come up to motivation. I will talk about pneumatic clamps. It’s a set of actuators which is used in many industries and in many applications.
Here you see 3 pictures from automotive production. It’s perhaps hard to see, but in all those pictures you see those bulky little clamps which are there to fix a steel, to clip or push some kind of materials. In a regular automotive production plant, there are several hundred, up to several thousand of those clamps or actuators which handle arbitrary movements or actions there.
首先我們來(lái)談?wù)勴?xiàng)目的動(dòng)機(jī)。我先介紹一下氣動(dòng)夾緊器。它是一組用于許多行業(yè)和許多應(yīng)用的執(zhí)行器。
在這里您可以看到 3 張來(lái)自汽車(chē)生產(chǎn)線的圖片。可能不是很容易分辨，但在所有這些圖片中，您會(huì)看到那些用來(lái)固定鋼材、夾住或推動(dòng)某種材料的笨重夾具。在常規(guī)的汽車(chē)生產(chǎn)工廠中，有數(shù)百至數(shù)千個(gè)用于處理任意運(yùn)動(dòng)或動(dòng)作的夾具或執(zhí)行器。
In this talk, the application focuses on a clamp – there are also other pneumatic actuators, but we will talk about the clamp, which is a very simple device in theory.
Here you see a detailed description of such a clamp, and in the end all the clamps do the same. It moves its clamping arms between 2 positions: open and close. This movement or the force it applies to the material then comes usually from a pneumatic valve.
In the lower part you see a pneumatic valve, which is usually connected to some kind of central compressor in a plant, which produces compressed air in the end with up to 5 bars or even higher. And the valve then either opens or closes, and the arm changes its position accordingly.
It's a very established technology, extremely robust technology, and also very cheap technology. Because the only moving part there is the valve, basically, which is quite robust and it's very simple to control. There is only a bit of information, basically, open or close, and nothing in between. You can apply strong forces to it because it has a stream of compressed air with several bars, and it’s also more or less maintenance free.
There are 2 major drawbacks in the application of those clamps, both related to pneumatics. One is, which we will not talk about today, that it’s a bit complicated to do the fine navigation, for example, if you change from one car model to the other one in the plant, then the layer of the sheets changes and arrangement is a bit complex, because this valve cannot store different positions – just the same position close and same position open. The other thing which is quite important, which is quite bad, is the energy efficiency. The energy efficiency is a big topic worldwide, and here in Germany it's even a bigger topic. The whole industry tries to get more efficient with energy consumption.
本次報(bào)告的案例主要圍繞氣動(dòng)夾緊器——當(dāng)然還有其他氣動(dòng)執(zhí)行器被廣泛應(yīng)用，不過(guò)我們將只討論夾緊器，它在理論上是一種非常簡(jiǎn)單的設(shè)備。
這是某個(gè)夾緊器的詳細(xì)描述，它會(huì)與其他很多夾緊器以相同功能運(yùn)作。它在兩個(gè)位置之間移動(dòng)其夾緊臂：即打開(kāi)和關(guān)閉。這種運(yùn)動(dòng)，抑或說(shuō)它施加到材料上的力通常來(lái)自氣動(dòng)閥。
圖片的下半部分是該氣動(dòng)閥，它通常連接到工廠中的某中央壓縮機(jī)，最終產(chǎn)生高達(dá) 5 巴甚至更高壓力的壓縮空氣。閥門(mén)打開(kāi)或關(guān)閉，夾緊臂就會(huì)相應(yīng)地改變其位置。
這是一項(xiàng)非常成熟的技術(shù)，非常強(qiáng)健的技術(shù)，也是非常經(jīng)濟(jì)的技術(shù)。由于唯一的活動(dòng)部件是閥門(mén)，它非常穩(wěn)固并且易于操控。它承載的信息量很小，即打開(kāi)或關(guān)閉，此外沒(méi)有任何其他信息。它可以承載強(qiáng)大的力量，因?yàn)槠鋷в腥舾砂偷膲嚎s空氣，并且它通常免于維護(hù)。
不過(guò)這類氣動(dòng)夾緊器有兩個(gè)弊端。其中一個(gè)缺點(diǎn)并非是今天的主題，即精確導(dǎo)航做起來(lái)有點(diǎn)復(fù)雜，比如在生產(chǎn)工程中從一個(gè)車(chē)型換到另一個(gè)車(chē)型時(shí)，由于氣動(dòng)閥不能存儲(chǔ)不同的位置——只能以相同的位置關(guān)閉和打開(kāi)，重新部署將會(huì)變得很復(fù)雜；另一件非常重要并且非常糟糕的問(wèn)題是能源效率。能源效率是世界范圍內(nèi)的一個(gè)重要議題，在德國(guó)更是一個(gè)焦點(diǎn)話題。 整個(gè)工業(yè)界都在想方設(shè)法提高能源使用效率。
Here is an interesting picture. On the left side, you see a chart about the total energy consumption, but only for compressed air. It is not the total energy consumption of the industry but only the part which goes into compressed air applications. And on the right side you see a chart which compares pneumatic actors to electric driven actors.
The actor which we have just seen at the moment is the left one. And it goes like that, you have a central compressor, which is usually a central compressor for large part of the plant. You need pretty high currents and high voltages there so it consumes a lot of electrical energy and then it outputs compressed air, which goes to all parts of the plant via wirings.
And a compressor has, in the best case, an efficiency of up to 20%, so you lose about 80% of the energy already during compression and transportation of the air for the plant through pipes. Then you have the next device which I've just shown, the electric valve together with the clamping unit, where you again lose about ? energy, so you end up only utilizing 5% of the energy you put in initially – only 5% energy is transformed into force applied to your process.
On the right side you see the picture that if we do not use compressed air but electrical actuators, then you need one DC-DC charger, which has an efficiency of about 80% and then you also have a motor and gears, but you end up keeping on 60% of the total energy. It also brings other benefits: you need no high voltage to power the system, so it can be installed decentralized, for example with solar cells on top of the plant, and many other benefits.
This is basically the driving force why the industry is looking to shift from pneumatic to electric in many applications, because if you compare the 5% energy efficient with 60%, it's a factor of 12 – 12 times of the power you can use in your process from the energy you spent already.
這里有幾張很有意義的圖片。最左側(cè)是一張關(guān)于總能耗的統(tǒng)計(jì)圖表，但僅限于壓縮空氣。即并非工業(yè)生產(chǎn)中的總能源消耗，而只是純粹用于壓縮空氣的這部分能源消耗。而在右側(cè)，您會(huì)看到一張對(duì)比圖，比較了氣動(dòng)執(zhí)行器和電動(dòng)執(zhí)行器。
我們剛才看到的設(shè)備就是圖中偏左這個(gè)氣動(dòng)執(zhí)行器。這張圖表達(dá)了，首先我們有一個(gè)中央壓縮機(jī)，它通常是服務(wù)于工廠大部分設(shè)備的中央壓縮機(jī)。為此需要相當(dāng)大的電流和電壓以輸出壓縮空氣，因此它會(huì)消耗大量電能，然后這些壓縮空氣被輸送到工廠的各個(gè)環(huán)節(jié)。
問(wèn)題在于，即使在最佳工況下的壓縮機(jī)，其效率最高也只能達(dá)到 20%，即單單在壓縮和通過(guò)管道輸送壓縮空氣的過(guò)程中，已經(jīng)產(chǎn)生了約 80% 的能量損失。然后在下一個(gè)生產(chǎn)環(huán)節(jié)，即我剛剛展示氣動(dòng)閥和夾緊器裝置，又將產(chǎn)生大約 ? 的能量損失，這樣一來(lái)初始的能源投入中只有約 5% 最終被有效利用——只有 5% 的能量是轉(zhuǎn)化為生產(chǎn)力。
最右側(cè)圖片展示了，如果我們不使用壓縮空氣、而是使用電動(dòng)執(zhí)行器完成生產(chǎn)的能源利用過(guò)程，為此我們需要一個(gè)直流變壓器，其效率約為 80%，此外還有一個(gè)電機(jī)和傳動(dòng)裝置等設(shè)備，在這部分處理操作完成以后，依舊保有60% 的總能量。它還帶來(lái)了其他好處：該系統(tǒng)不再需要高壓驅(qū)動(dòng)，因此可以采用分布式部署，如在工廠頂部安裝太陽(yáng)能電池；此外這種設(shè)計(jì)還有諸多其他優(yōu)勢(shì)。
這也是工業(yè)生產(chǎn)中很多應(yīng)用中從氣動(dòng)逐漸轉(zhuǎn)向電動(dòng)的主要驅(qū)動(dòng)力，因?yàn)槿绻麑?5% 的能效與 60% 的能效進(jìn)行比較，這是 12 倍系數(shù)的變革——12 倍的能效提升。
As already said, the lower part here, which is currently a pneumatic valve, will be replaced – and in the application I’m talking about it was already replaced – by an electric motor. The picture on the right shows such electric motors, which are built already in a specific way to have about the same size and to be installable in this kind of actuators.
There are several differences already between having a valve and having a motor. A valve, as I said, is very simple, which you have only a bit of control signal, basically, which is either valve open or valve close. A motor is a very complex component compared to the valve, and in this application usually brushless DC motors are used. You need a motor controller which controls the motor already, and then you have a set of sensors in the motor which are Hall sensors and also encoders, which give you back information of the motor.
So you have quite a lot more of information compared to the valve which has only a bit – you only say “open” or “close” without knowing if the valve really is open or closed, you just think it is like that because of the sheer force that the pneumatic clamp brought in, but you are not certain.
如前述，該設(shè)備下半部分當(dāng)前是一個(gè)氣動(dòng)閥，它將被電動(dòng)機(jī)取代——在我們的實(shí)踐中已經(jīng)被全面取代。右圖顯示了作為改進(jìn)措施的電動(dòng)機(jī)，它們被專門(mén)設(shè)計(jì)制造，具有大致相同的尺寸，可以安裝在這種執(zhí)行器中。
氣動(dòng)閥和電動(dòng)機(jī)之間存在不小的差異。之前介紹過(guò)，氣動(dòng)閥的功能非常簡(jiǎn)單，只承載了非常簡(jiǎn)單的控制信號(hào)，即閥門(mén)打開(kāi)或閥門(mén)關(guān)閉。而與氣動(dòng)閥相比，電動(dòng)機(jī)是一個(gè)非常復(fù)雜的組件，在我們的實(shí)踐中通常使用無(wú)刷直流電機(jī)。首先需要一個(gè)電機(jī)的控制器，并且電機(jī)中有一組傳感器，即霍爾傳感器和編碼器，以提供電機(jī)的反饋信息。
因此，與只氣動(dòng)閥相比，電動(dòng)機(jī)承載了更多更復(fù)雜的信息——對(duì)氣動(dòng)閥來(lái)說(shuō)，我們只給出了“打開(kāi)”或“關(guān)閉”的輸入信號(hào)，卻不知道其是否真的真的被打開(kāi)或關(guān)閉，我們只是根據(jù)氣動(dòng)夾緊器的運(yùn)動(dòng)判斷它執(zhí)行了這個(gè)操作。
02 Data Collection 數(shù)據(jù)采集架構(gòu)
Now we talk about the data coming out of this application and the architecture we have applied there.
In such an automotive power plant where we have the application, we have about 10,000 of those pneumatic actors (clamps), which are replaced now by motors – all the valves were taken out and replaced by motors, then we end up with 10,000 motors in the plant. Each motor generates currently about 10 different measurements, which will be more in the future, including voltage, current, torque, angles, etc. Currently, they operate at 1,000 Hz so we have one sample of all those 10 measurements each millisecond.
The initial idea was to just install powerful database Apache IoTDB in our case. Then we did some tests and did the math, and the raw volume of data – if we would just send over all those signals from all 10,000 motors, we would end up with an incoming data stream to the server of about 6.1 Gbit (per second). The server could still handle, I guess, but the network could not handle it, because it is in a plant and the network in plants are not that powerful as networks in data centers.
We have in the best case scenario, 1 Gbit/s network and in worst case scenario, we might even only have 100 Mbit/s network bandwidth, so the simple approach does not work.
接下來(lái)我將介紹一下在我們實(shí)踐中的數(shù)據(jù)來(lái)源以及我們?cè)O(shè)計(jì)的架構(gòu)。
在我們這個(gè)汽車(chē)生產(chǎn)線的項(xiàng)目中，有大約 10,000 個(gè)氣動(dòng)執(zhí)行器（夾緊器），現(xiàn)在已被電動(dòng)機(jī)所取代——所有的氣動(dòng)閥都被取出并替換為電動(dòng)機(jī)，即生產(chǎn)線上最終有 10,000 個(gè)電機(jī)。每個(gè)電機(jī)有約 10 個(gè)不同測(cè)點(diǎn)，并且測(cè)點(diǎn)數(shù)將來(lái)會(huì)更多，包括電壓、電流、扭矩、傾角等。目前它們以 1,000 Hz 的頻率運(yùn)行，因此我們每毫秒從所有這 10 個(gè)測(cè)點(diǎn)獲取一個(gè)樣本。
最初的想法只是將功能強(qiáng)大的時(shí)序數(shù)據(jù)庫(kù) Apache IoTDB 應(yīng)用在我們的案例中。然后我們做了一些測(cè)試并計(jì)算了原始數(shù)據(jù)量——如果我們發(fā)送來(lái)自所有 10,000 個(gè)電機(jī)的所有信號(hào)，最終會(huì)向服務(wù)器輸入大約 6.1 Gbit（每秒）的數(shù)據(jù)流。也許服務(wù)器尚能處理這個(gè)數(shù)據(jù)量，但網(wǎng)絡(luò)帶寬無(wú)法承受，因?yàn)閿?shù)據(jù)來(lái)自工廠的生產(chǎn)線，而工廠中的網(wǎng)絡(luò)并不如數(shù)據(jù)中心的網(wǎng)絡(luò)強(qiáng)大。
工廠中最好網(wǎng)絡(luò)條件約有 1 Gbit/s 的帶寬，而最壞的情況下，可能只有 100 Mbit/s 的帶寬，所以最簡(jiǎn)單的 IoTDB 部署是行不通的。
So we ended up with a 2-layer approach, and for other reasons there are already gateways installed. About 20 motors are controlled by one gateway, and on these gateways we can install instance of IoTDB, then they can run locally.
In this scenario, we have 500 tags – 10,000 motors overall, but 20 connected to one gateway. On all those gateways, we put an instance of IoTDB, then we can store the data there.
所以我們最終采用了 2 層結(jié)構(gòu)，由于一些其他原因，生產(chǎn)線上已經(jīng)安裝了網(wǎng)關(guān)設(shè)備。每個(gè)網(wǎng)關(guān)控制 20 個(gè)電機(jī)，在這些網(wǎng)關(guān)上安裝 IoTDB 副本，就可以在本地運(yùn)行了。
在這種情況下，我們將有 500 個(gè)測(cè)點(diǎn)——共 10,000 個(gè)電機(jī)，但每 20 個(gè)連接到一個(gè)網(wǎng)關(guān)。我們?cè)谒羞@些網(wǎng)關(guān)上，部署了 IoTDB 服務(wù)器并存儲(chǔ)數(shù)據(jù)。
And you see, the raw volume of data for each of those edge gateways from 20 sensors we measure is about 12.2 Mbit/s, so it's easily handleable.
Now we get all data stored, that is nice, but to work with the data that's not good, because we need to connect to 500 databases to do things with the data. We want to have the data in a central database, so we set up another central database and used IoTDB’s sync protocol to just forward the data to the central server.
As we have done it via IoTDB sync protocol, what we already observed is, we get a nice reduction of data we need to transfer from over 6 Gibt/s to only 1.2 Gbit/s, because we are not transferring raw data but we are transferring TsFile in the end, which are highly compressed data sets. So in our real use case scenario, we saw a reduction of factor 5, which could even be more or could also be less, depending on what kind of data what we have.
In our case, we ended up with 1.2 Gbit/s, which is way better of course, but it's still too much to handle in our scenario.
如圖所示，每個(gè)邊緣網(wǎng)關(guān)的原始數(shù)據(jù)量約為 12.2 Mbit/s，這樣數(shù)據(jù)處理就簡(jiǎn)單得多。
現(xiàn)在我們已經(jīng)存儲(chǔ)了所有數(shù)據(jù)，這固然很好，但是要處理所有數(shù)據(jù)就不太方便了，因?yàn)槲覀冃枰B接到 500 個(gè)數(shù)據(jù)庫(kù)來(lái)完成數(shù)據(jù)處理。我們希望將數(shù)據(jù)存放在中央數(shù)據(jù)庫(kù)中，因此我們?cè)O(shè)置了另一個(gè)中央數(shù)據(jù)庫(kù)并使用 IoTDB 的同步協(xié)議將數(shù)據(jù)轉(zhuǎn)發(fā)到中央服務(wù)器。
正如我們通過(guò) IoTDB 同步協(xié)議實(shí)現(xiàn)的效果，我們已經(jīng)觀察到，需要傳輸?shù)臄?shù)據(jù)量從超過(guò) 6 Gbit/s 減少到僅 1.2 Gbit/s，這是因?yàn)槲覀儾](méi)有傳輸未經(jīng)處理的原始數(shù)據(jù)，而是高效壓縮后的TsFile 數(shù)據(jù)文件。因此，在我們的真實(shí)生產(chǎn)場(chǎng)景中，數(shù)據(jù)量被縮小了5倍，當(dāng)然這個(gè)數(shù)值可能更大也可能更小，具體取決于應(yīng)用場(chǎng)景中的數(shù)據(jù)類型。
在我們的實(shí)踐中，數(shù)據(jù)量已經(jīng)被壓縮到1.2 Gbit/s，這與原始數(shù)據(jù)量相比當(dāng)然已經(jīng)很好，但在處理起來(lái)依舊有些困難。
So, what we did in the end? As I said, every motor generates about 10 different measurements and in the discussions with the customer, we saw that not all are equally important. Some are important and they want to do analysis on them, and others are just there to be stored in the first and later they might use them for analytics.
So what we did is we reconfigured our edge databases to store all those data, but only some of those data points are forward to the central server. We ended up with only taking 2 of those 10 measurements, which we considered are important that we want them to be always live available, and this is what I called “hot path”.
These data are basically live available for all the 10,000 devices on the central server. And all the other data points are stored on the edge IoTDB servers, but are not synced automatically. We either leave them on the edges or, from time to time, we transfer them over by a hard disk – just go to the edge server and take out the data, then go to the central server and export there. Because the problem is not the amount of data just came in, but is really just the bandwidth of the network, which is not enough to transfer the data live.
So this is the scenario that we ended up with, with the decision to take 2 out of those 10 values to a central server, and we ended up with a network bandwidth of about 250 Mbit/s we needed for our application, then the network is still available for other applications. So this is the scenario we have taken here.
This has both benefits from our perspective: on one hand, we get all data stored; and on the other hand, for data analytics you don't need to connect to 500 databases to gather the important data but only to 1 central server.
那么，我們最后又做了什么改進(jìn)呢？如之前描述，每個(gè)電機(jī)都有約 10 個(gè)不同測(cè)點(diǎn)，在與客戶的討論中，我們發(fā)現(xiàn)并非所有測(cè)量值都同等重要。有些很重要，客戶想對(duì)它們進(jìn)行分析處理，而另一些只是先存儲(chǔ)，以備日后不時(shí)之需。
因此，我們重新配置了邊緣版數(shù)據(jù)庫(kù)以將所有數(shù)據(jù)保存，但只有其中一部分?jǐn)?shù)據(jù)點(diǎn)會(huì)被傳輸?shù)街醒敕?wù)器。我們最終只選取了這 10 個(gè)測(cè)點(diǎn)中的 2 個(gè)最重要的、需要保持始終可用的實(shí)時(shí)數(shù)據(jù)的測(cè)點(diǎn)，這即是所謂的“熱路徑”。
對(duì)于中央服務(wù)器來(lái)說(shuō)，來(lái)自所有 10,000 臺(tái)設(shè)備的這兩個(gè)測(cè)點(diǎn)的數(shù)據(jù)都是實(shí)時(shí)可用的。相對(duì)的，所有其他的數(shù)據(jù)點(diǎn)都只被存儲(chǔ)在邊緣 IoTDB 服務(wù)器上，而不會(huì)被自動(dòng)同步。我們要么將這些數(shù)據(jù)保留在邊緣側(cè)，要么根據(jù)需求通過(guò)磁盤(pán)將它們傳輸——只需到邊緣服務(wù)器取出數(shù)據(jù)，然后導(dǎo)出到中央服務(wù)器。因?yàn)橹饕獑?wèn)題不在于獲得的數(shù)據(jù)量，而只在于網(wǎng)絡(luò)帶寬的局限，其不足以實(shí)時(shí)傳輸所有數(shù)據(jù)。
所以這就是我們最終的場(chǎng)景，即從這 10 個(gè)測(cè)點(diǎn)數(shù)據(jù)中取出 2 個(gè)實(shí)時(shí)同步到中央服務(wù)器，在我們的實(shí)踐中最終所需大約 250 Mbit/s 的網(wǎng)絡(luò)帶寬，其余帶寬仍然可用于其他目的。這就是我們實(shí)踐中的最終方案。
從我們的角度來(lái)看，這有兩個(gè)好處：一方面，我們存儲(chǔ)了所有數(shù)據(jù)；另一方面，進(jìn)行數(shù)據(jù)分析時(shí)，我們不需要連接到 500 個(gè)數(shù)據(jù)庫(kù)來(lái)收集重要數(shù)據(jù)，而是連接到一個(gè)中央服務(wù)器即可。
03 Summary 項(xiàng)目總結(jié)
I want to give you a short summary of what I have just talked about and shown you. As said already, the things I've shown here are from a German automotive OEM’s production plant, and will soon be rolled out also to other OEMs. Interesting takeaways are, generally speaking, this trend to improve the energy efficiency of production plants, there is general change from pneumatic actors to electric actors, and this leads to a massive increase in the amount of data that is generated, because motors just generate way more data that can be used for more fine-grained control for optimizations.
One thing to keep in mind is, this is something we see especially in Germany, because in Germany the IT infrastructure of many production plants are not top, and we have to keep an eye on the on the available network bandwidth. This is something we learnt and definitely can make sense of one IoTDB instance running on the edge and another one running somewhere else just doing synchronization, because then the data is already transferred via highly compressed TsFiles, basically out of the box. Another takeaway that I would just talk about is, in our experiments we saw a compression down to 20% of the raw volume of network bandwidth – we have some overhead, but this is a benefit about a factor of 5. So with this compression ratio you only need 20% of the original network bandwidth.
And all the architecture I've shown you basically involved no change in a single line of code of the systems importing data or the systems using the data, because all we need to change there are configuration settings of which IoTDB instances to be connected to, or (in IoTDB instances) the configuration settings for the sync server and sync receiver functionality, so all of these is carried out without that somebody has to change a single line of code.
And another thing I’ve just talked about was, we split up the data in two paths: the hot data, which is centrally available for each motor, is basically live because it was synced all the time; and the cold data which is stored, but on edge devices and the process to gather the data is a bit more complicated because you need to connect to these instances, and then browse the data there or to manually transfer it to the central server.
And another observation which we have done in the process is that – I don't know if this has already ever been investigated – the sync server protocol scales pretty well with the amount of servers that they send the data, in our case of 500 databases to one single sync receiver, and it works quite well.
This is what I wanted to tell you about our use case.
最后我將對(duì)剛才介紹的案例做個(gè)總結(jié)。如前述，這是在一家德國(guó)汽車(chē)制造商的生產(chǎn)工廠中進(jìn)行的實(shí)踐，并且很快也會(huì)推廣到其他制造商。有趣的是，一般來(lái)說(shuō)，這是提高生產(chǎn)工廠能源效率的大趨勢(shì)，即從氣動(dòng)執(zhí)行器到電動(dòng)執(zhí)行器的轉(zhuǎn)型，而這會(huì)導(dǎo)致生產(chǎn)線中的數(shù)據(jù)量大量增加，因?yàn)闉榱藢?shí)現(xiàn)流程優(yōu)化與精細(xì)化控制而使用的電動(dòng)機(jī)本身會(huì)產(chǎn)生更多數(shù)據(jù)。
要知道，在德國(guó)，這個(gè)案例中遇到的問(wèn)題是非常典型的情況，因?yàn)榈聡?guó)許多生產(chǎn)工廠的 IT 基礎(chǔ)設(shè)施都不是頂級(jí)的，因而我們必須密切關(guān)注可用網(wǎng)絡(luò)帶寬。這是我們的經(jīng)驗(yàn)，驗(yàn)證了可以通過(guò)一個(gè)運(yùn)行在邊緣的 IoTDB 和另一個(gè)運(yùn)行在其他地方的 IoTDB 完成數(shù)據(jù)同步，因?yàn)閿?shù)據(jù)通過(guò)已經(jīng)高度壓縮的 TsFile 傳輸，是開(kāi)箱即用的。另一個(gè)要點(diǎn)是，在我們的實(shí)踐中，壓縮比下降到原始網(wǎng)絡(luò)帶寬量的 20%——新方案確實(shí)帶來(lái)了其他的開(kāi)銷(xiāo)，但完成了大約 5 倍的提升。即在這個(gè)壓縮率下，只需要原始網(wǎng)絡(luò)帶寬的 20%。
值得一提的是，我所展示的所有架構(gòu)不會(huì)涉及導(dǎo)入數(shù)據(jù)的系統(tǒng)或數(shù)據(jù)應(yīng)用系統(tǒng)的任意一行代碼的變動(dòng)，因?yàn)槲覀冃枰牡闹皇且B接到哪些 IoTDB 的配置，或者（在 IoTDB中）同步服務(wù)器和同步接收器功能的配置，而所有這些配置的變動(dòng)都不會(huì)影響任何原始代碼。
我剛才談到的另一件點(diǎn)是，我們將數(shù)據(jù)分為兩條路徑：一是熱數(shù)據(jù)，每個(gè)電機(jī)集中可用的熱數(shù)據(jù)基本上是實(shí)時(shí)的，因?yàn)樗恢痹诒煌剑?二是冷數(shù)據(jù)，只被存儲(chǔ)在邊緣設(shè)備上，對(duì)冷數(shù)據(jù)的處理會(huì)有些復(fù)雜，因?yàn)樾枰B接到這些客戶端，然后在那里調(diào)用數(shù)據(jù)或手動(dòng)將其傳輸?shù)街醒敕?wù)器。
我們?cè)谶@個(gè)過(guò)程中所做的另一個(gè)發(fā)現(xiàn)是——我不知道這是否已經(jīng)被提起過(guò)——同步服務(wù)器協(xié)議可以很好地?cái)U(kuò)展，在我們有 500 個(gè)數(shù)據(jù)庫(kù)的情況下，數(shù)據(jù)可以被完好并且穩(wěn)定的發(fā)送到一個(gè)同步接收器。
以上就是我想和大家分享的應(yīng)用實(shí)例。
If you have any questions leftover, you could find my contact information here. As I already said, I'm working for the company Pragmatic Industries which I founded, and I’m also a long time Apache IoTDB PMC member, and I am also working with Timecho Europe to bring more of those use cases to European industry. If you have any questions, just Ping me, thank you.
如果您還有任何疑問(wèn)，可以在這里找到我的聯(lián)系方式。正如一開(kāi)始介紹的，我在我創(chuàng)辦的 Pragmatic Industries 公司工作，同時(shí)也是 Apache IoTDB PMC 的長(zhǎng)期成員。此外，我還致力于通過(guò) Timecho Europe 將 IoTDB 更多的應(yīng)用到歐洲的工業(yè)實(shí)踐中。如果您有任何問(wèn)題，請(qǐng)和我聯(lián)系，謝謝！
更多內(nèi)容推薦：
? 了解更多 IoTDB 應(yīng)用案例
? 回顧 IoTDB 2022 大會(huì)全內(nèi)容
