Cyber Physical System (CPS) can be viewed as a large networked system, distributed geographically with each CPS site having hybrid computing patterns. Because of the collective might of CPS nodes, the capabilities of CPS could have far reaching effects in finding solutions to grand challenges in fields that combine science, engineering, and economics. Thus, CPS applications will require resources from different domains. In the absence of central authority to moderate and manage resource requests for CPS services, there is a need to facilitate the coordination of services and cooperation to share resource knowledge among resource providing nodes in CPS. There is a large variety of heterogeneous data stored in CPS nodes, includes traditional relation database that from different software vendors, including different types of NoSQL database, and different types of file, all of the data resources grow rapidly, become a great repository of data. For meeting the requirements of knowledge services, we need to extract the knowledge implied in heterogeneous data, represent and integrate the knowledge to support knowledge system. Knowledge Base Construction (KBC), the process of populating a knowledge base with semantic information from heterogeneous data, is an important and difficult task. In recent years, several large-scale knowledge bases have been constructed, such as YAGO[1], NELL[2], DBpedia[3], IBM’s Watson[4] and Microsoft’s EntityCube[5]. However, the rapid growth of huge heterogeneous textual data, stored in different data sources with different format and structure (e.g. mobile computing, social computing, Internet of Things, etc.), make it urgent to find a way to acquire and management knowledge from these data quickly. It is clear that building a distributed parallel framework to extract knowledge and construct knowledge base meet this requirement efficiently. Simultaneously, the present expression of heterogeneous textual data is weak semantic for random distribution so that it is not conducive to Parallel and Distributed Computing (PDC). It is obvious that there is a demand for a good data representation method with strong semantic can help parallel extraction of knowledge for better performance.