Smart home giant Aqara builds IoT microservice platform based on KubeSphere

Posted May 27, 20209 min read

Background

From traditional operation and maintenance to containerized Docker Swarm orchestration, from Docker Swarm to Kubernetes, and then run the SpringCloud microservices family bucket on Kubernetes, and finally embrace KubeSphere, and build Lumi Lianchuang's own IoT microservices platform based on KubeSphere. Milestone has been running KubeSphere and Kubernetes in a production environment for more than half a year, and has accumulated rich experience in microservice application development and application platform operation and maintenance. This article was contributed by Wei Hengsheng and Xu Yangbing, operation and maintenance engineers of Shenzhen Lumi Lianchuang Technology Co., Ltd. The image material is from the official website of Aqara( https://www.aqara.com/ ).

Aqara Introduction

Shenzhen Lumi Lianchuang Technology Co., Ltd.(referred to as "Lumi Lianchuang", the official website https://www.aqara.com/ ) was established in 2009 and is headquartered in Shenzhen, covering ultra-low power wireless sensors, Zigbee wireless network technology, smart home gateway edge computing technology, algorithms and AI, platform opening and access capabilities, etc. In 2016, Shenzhen Lumi Lianchuang Technology Co., Ltd. launched its own brand Aqara with the concept of whole house intelligence (Aqara is derived from the Latin acutula, meaning smart, ARA means to go home, Aqara combining the two means The family is becoming more and more intelligent). Aqara is committed to building a smarter life for users through a series of smart home product technologies and service provider models. Aqara's products include various sensors such as temperature, humidity, doors and windows, human body, water immersion, smoke, gas, light and sleep, as well as various types of intelligence such as smart switches, sockets, curtain motors, air conditioning controllers, dimmers, door locks, etc. The controller currently supports both automated control and big data analysis platforms for industry applications.

Adhering to the vision of leading the Internet of Things technology and serving thousands of households , Aqara adheres to the mission of persistently pursue user experience and unremittingly create user experience . It constantly innovates in the smart home industry and eventually becomes a leading brand in the industry.

From traditional operation and maintenance to container technology

As soon as the operation and maintenance are deep, Wei Hengjun, as a senior operation and maintenance engineer with many years of work experience, has carried the network cable pliers and installed the operating system numbly from the initial carrying the machine to the computer room, and arbitrarily installed the operating system. Programs and debugging application services, and waking up with a series of warnings during that night, always feel like a great firefighter.

Alt text

Data Center(picture from Unsplash)

The rapid iterative update of technology has ushered in microservices, ushered in virtualization technology, and also ushered in containerized and cloud-native technologies. O & M has also evolved from the initial human O & M to scripted O & M, to platform O & M, and finally to container O & M. The machines I have operated and maintained have unconsciously maintained dozens of servers to nearly a thousand servers. The traditional application deployment method requires a lot of time to prepare configuration files and operation precautions for each iteration. , Databases, etc., and then go through the approval of a group of people, and then send it online. This period has passed half a month. In this era of Internet speed, it is obvious that the disadvantages of this traditional method are very obvious, and the container should be up to date. Born.

Use Docker Swarm to build a container orchestration system

The traditional way of deploying applications has very low resource utilization, and the length of time has caused the bosses to grind their teeth. In this case, I started to contact the container in 2017, trying to develop and test the environment on the company. At that time, the resource utilization rate of the company's development and test environment was increased by 50%. By 2018, Docker Swarm was used to arrange containers in the production environment, which significantly improved resource utilization.

Alt text

From the command line to the scripting, and finally to the platform, it is hard and difficult. When I first joined the Green Rice family, I found that the operation and maintenance of Green Rice was still in the original savage stage. Looking back at four weeks, I could only repeatedly analyze the situation with my sleeves under pressure, and found that more than 80%of the micro service architecture of Green Rice is partial memory. Type of service, resource utilization is very low, especially CPU, disk storage, very annoying. And the speed of iteration is not satisfactory. Meditation, decided to change this situation. From continuous integration, Jenkins, Harbor build, to test environment Docker Swarm layout. This has greatly improved the delivery speed and delivery quality of the test environment, but slowly found that the business volume has risen too fast, and the Docker Swarm scheduling disadvantage is obvious:

  • Poor cross-platform support;
  • During the peak period of traffic access, there will be a timeout problem during internal Service communication

From Docker Swarm to Kubernetes

The era of three carriages is a thing of the past, and Kubernetes beat Docker Swarm and Mesos to become the de facto standard in the field of container orchestration. Therefore, our business architecture has shifted from Docker Swarm to Kubernetes. Choosing Kubernetes took root in my heart a few years ago, especially when there is a need to operate and maintain nearly a thousand machines recently. An operation and maintenance friendly and unified container cloud platform has become our just-needs based on kubernetes large-scale landing cloud native microservices applications.
Alt text

Selection of Open Source Container Platform:Embrace KubeSphere

However, for the native installation and operation and maintenance of Kubernetes, we still use third-party open source solutions. After repeated pondering, we finally chose to use third-party open source projects. It looks like Rancher and KubeSphere have become options for consideration.

KubeSphere is an open source project initiated by Qingyun QingCloud and co-developed with multiple companies. Compared with Rancher and KubeSphere, the latter not only has a refreshing user interface, but also a wizard-style resource creation method, which is completely application-centric, and is more inclined to the management of Kubernetes cluster resources, provides an elegant API interface, and is integrated and packaged on top of Kubernetes In addition to the commonly used functional components for our O & M development, such as Jenkins, Harbor, Promethues, and Apache SkyWalking, it also supports deployment in any infrastructure environment, so we chose the KubeSphere container platform without hesitation.

Alt text

The compatibility of KubeSphere across multi-cloud platforms and the choice of supporting multiple plug-ins have deepened our understanding of the various modules of Kubernetes in the process of use and promoted the pace of Kubernetes container orchestration for the production environment. In addition, KubeSphere liberates the repetitive tasks we face in daily operation and maintenance, and reduces the overall maintenance cost of the application. It is a tool for operation and maintenance, and a boon for Internet companies.
Alt text

Lvmi IoT microservice platform deployment architecture

At present, the company mainly uses 7 servers to build the cluster on Tencent Cloud. The configuration specifications of the cluster machines are as follows.

Alt text

At present, all stateless services are running in KubeSphere. Stateful data storage services, we use Redis, HBase, Flink, Elasticsearch, MySQL and other cluster services on the cloud.

Up to now, it has been running for more than half a year and no major problems have occurred. This pushes us to plan to migrate all stateful and stateless services in the company's development, testing, and production environments to KubeSphere in the near future.

Alt text

Lumi IoT microservice platform design architecture

First of all, you can look at the business architecture diagram of Lumi Internet of Things. At present, the services of Lumi overseas are basically all running on KubeSphere, including Gateway microservice routing and scheduling, Push, Send push, iftt timing and so on.

Alt text
Since our business is dominated by Java, the Greenmi IoT microservice platform is based on the SpringCloud framework for microservices, using the Apollo distributed configuration center to manage configuration, and Eureka registry service registration and discovery.

Alt text
Combine Ribbon and Feign to achieve microservice load balancing and service invocation. At the same time, we use the Hystrix thread pool to achieve isolation, fuse and downgrade, sentinel current limiting, and the springcloud-gateway gateway routing is used to implement routing scheduling, the log uses a classic ELK combination, and APM uses SkyWalking as a Java microservices distributed system Application performance monitoring tool.
Alt text

As shown in the above figure, we use Tencent Cloud for IaaS. Platform(platform layer) is mainly microservices for IoT business platform. Most of the applications in Platform layer run on the KubeSphere container platform, and all sub-devices pass the Zigbee protocol. Connect Hub devices, namely smart gateways, smart socket gateways, cameras, etc. The Hub devices communicate with Lumi Smart Home's microservice platform through the RPC protocol. The microservice platform provides data for applications such as App and SaaS, and reverse applications pass a series of security Authentication and authentication call the green rice microservice platform to control smart home equipment. The service layer has plug-ins such as link tracking, basic monitoring, CI/CD, etc.

KubeSphere makes it easier for us to get started with Kubernetes, accelerates the launch of Kubernetes in the production environment, has a significant efficiency improvement for business iteration, and enables R & D to switch and deploy function modules for verifying various applications at will.

As of now, this set of IoT microservice platforms has been in production and operation for more than half a year in our Greenmi Lianchuang without major problems. Therefore, we plan to put all the stateful All stateless services are migrated to KubeSphere.

Q & A

Q:Have you encountered any problems during use?

A:Yes, such as DevOps pipeline to solve the issue of War/Jar package release. The DevOps pipeline not only needs to solve the problem of packaging the mirror to the mirror warehouse, but also needs to be compatible with the deployment method of the old business war package distributed through Ansible. At first, there was no solution.

After some research, I understand that the entire DevOps process is that the Jenkins-agent pulls the Pod corresponding to the template, and runs through the various processes of Pipline, but the problem comes again. After the maven Pod of the Java template is executed, it exits, but it is impossible. Get the compiled Jar package.

Finally, we found that you can log in to the Jenkins server, select Manage Jenkins => Configure System, find the corresponding template, and operate as shown in the screenshot. Specify mav package -Dpath = ${target \ _path} in Pipline to solve the above solution problem!

Q:What kind of application development platform can carry the future of smart home?

A:Perfect auditing, monitoring alarms, permission distribution, and can customize elegant resource expansion and contraction strategies, personalized customization of elegant plug-in plug-ins, self-examination strategy of the platform's own conventional problems, and clear and clear logs, good All of this is supported on the KubeSphere container platform.

Q:What functions or designs of KubeSphere still need improvement?

A:The recommendations are as follows:

  • The interface language switch is too hidden;
  • Granfana template integration flexibility can be a little more;
  • Kubernetes node expansion can be made simpler, and it is best to support interface node expansion.
  • Create pipeline to support copy from;
  • Running pipeline supports multi-select batch;
  • It is better to have some examples in the api documentation. In the current Swagger, the required parameters of many interfaces cannot be read, and the readability is not very good.

Postscript

Thank you very much for the practical implementation and sharing of the IoT microservice platform in the smart home industry brought by the two users of Greenmi Link! From traditional operation and maintenance to containerized Docker Swarm orchestration, from Docker Swarm to Kubernetes, and then running SpringCloud microservices family bucket on Kubernetes, to finally embrace KubeSphere, and build Lumi Lianchuang's own IoT microservices platform based on KubeSphere, this It is also the evolution process of the application microservice platform of some domestic enterprises.

If you are very interested in the detailed implementation of the implementation of the IoT microservice platform of Lumi Lianchuang, I hope to learn more and communicate with the two engineers. Welcome to join the KubeSphere open source community exchange group. We will invite two engineers to do an online live technology sharing for everyone according to your needs. In addition, if you want to share the practice of KubeSphere and Kubernetes in your enterprise environment, we also welcome your submission!

For more information, please visit www.qingcloud.com