Introduction

Web usage mining (WUM) also known as web log mining is the application of data mining techniques applied on web data to extract relevant data and discover useful patterns [1], with the aim of improving the usefulness of the various web based applications. The process of web usage mining can be broadly divided into four phases—sourcing or collection of data, pre-processing or removal of ‘noise’, discovery of interesting patterns and the last analysis of the discovered patterns [2]. The first phase is simply sourcing of the data or information that is to be processed from various resources— which in our case will predominantly be the log files obtained from the Web servers, Web Proxy Servers and Client Browsers [3]. The quality of source log file is improved in the second phase by removing the extraneous, immaterial data termed as ‘noise’, to make the log file ready for further processing like segregating it into user and ‘sessions’. Identification [4–6]. In the third stage, various statistical techniques like ‘association’, ‘classification’ and ‘clustering’ are applied to the pre-processed data to discover interesting arrangements or patterns [7–9]. In the last stage the identified patterns are subjected to various analytical tools and mechanisms [1, 10] to finally extract the sublimate, the ‘essence’ or knowledge which has applications in a wider variety of fields like commerce, improvement of web based applications, identification of criminals and international security, attracting new customers and their retention, increasing website visits etc. Present age, is rightly referred to as the age of information and knowledge, as having useful information/knowledge at ‘right time’ gives one a huge advantage over others leading to appropriate and efficient decision making and plan execution. But in the last two decades, with the advent of internet technology and open source resources, there has been a humongous surge in the amount of information leading to ‘information overload’. It becomes a challenging and time consuming task to sift through this huge volume of data to extract relevant information on a topic. To overcome this issue, certain techniques have been developed, which helps us to retrieve relevant results efficiently and accurately from the web. The plethora of these data mining techniques, methodologies, algorithms which are applied on the web data and web logs to extract relevant data and discover useful patterns, with the aim of improving the usefulness of the various web based applications is known as web usage mining. Ontology is an explicit formal specification of the terms in the domain and relations among them [11]. Commonly it is defined to consist of abstract concepts and relationships only. In some rare cases, ontologies are defined to include instances of concepts and relationships [11]. In this paper it has been shown that how web log data can help to continuously update and improve the knowledge base of the existing ontology from time to time. Web mining techniques can be applied on web log files to find some suggestions to improve the existing ontology and some research has already been done in this field [11–16]. In this work the researcher has used prote´ge´ 5.0 (for ontology construction) and weka tool (for data mining algorithms). The thrust of this paper is facilitating information retrieval through a novel ontology management approach based using web log data.

The process of finding out valuable knowledge drawn out from web data is known as web mining. Identifying the various patterns and utilizing the vast knowledge extracted from those patterns is important from various perspectives such as business intelligence, e-learning, personalization etc. The web mining area which deals with extraction of patterns from user’s weblogs is called as web usage mining, which is an implementation part of data mining. This paper focuses on the working of web usage mining, data sources for web usage mining and applications of web usage mining is explained in detail in this paper. Further, we explain the issues and current challenges in web usage mining.

INTRODUCTION

With the advent of various technologies, social networking sites, e-business site, e-learning sites etc. Every second new data is generated. The data growth rate is exponential. On a single tweet, for example, one can find millions of messages or retweets on Twitter. All this is possible because of the internet. Thus, The World Wide Web is a large source of varied data, a collection of billion documents, which either comes from the content in the web pages or from the various hyperlinks or structure of the websites i.e. web structure or from the log files depicting the web usage. The data or information is collected from various points or data sources. The Internet is also a form of network and in a network data is available at various nodes. Here, nodes can represent servers, client machines, intermediate devices popularly called as proxy servers or various databases that are stored on machines. Web mining is the part of data mining from which we derive meaningful and useful knowledge from text or contents present in web pages, hyperlinks and user usage logs [2]. For a clearer understanding, web mining has 3 parts: Web Structure Mining, Web Content Mining, and Web Usage Mining. The Web Content mining deals with the raw data available in the web pages; the source data mainly consists of the textual information, images, graphic, audio etc. present in the web documents. Since web content mining is basically related to the web content, the main application fields of web content are a content-based ranking of web pages and contentbased categorization [3]. The main focal point of Web Structure mining is the structure of websites. The way in which a web page is arranged is outlined by the structure data. There are two types of structure information: Inter-page structure information and Intra-page structure information. The intra-page structure information is illustrated by a treelike structure which is composed of HTML and XML tags. [3,9] whereas the hyperlinks that associate one page with another is inter-page structure information. With the information obtained from web structure mining, link-based categorization of web pages is done; this is an application of web structure mining. Web Usage Mining is the portion that focuses on extraction on knowledge from log files and helps in finding user behavior [13,15]; sources for these log files is servers, proxy servers or clients which can collect the typical information such as IP address, page references, access time etc. which are represented into standard formats and major application areas are web personalization, Business intelligence, E-commerce, E-learning etc. The paper is detailed as Section I defines web mining and its types; Section II illustrates web usage mining, Section III discuss the process of Web Usage Mining and the various data sources. Section IV describes the different techniques used in the process of web usage mining. Section V discusses the application areas in detail. Section VI discusses the trending issues and challenges faced in this field. Section VII discusses the future trends in web usage mining.

1- INTRODUCTION

2- DATA ACQUISITION STRATEGIES

3- FOCUSED CRAWLER: AGRICULTURAL DATA

4- INCREMENTAL CRAWLER: CORN PRICE DATA

5- CUSTOM CRAWLER: E-COMMERCE DATA

6- CONCLUSIONS

REFERENCES

Abstract

Agricultural big data can be used to guide agricultural production, forecast agricultural market demands, and support agricultural decisions. How to effectively extract and use the information on the Internet, which contains a large amount of agricultural information, has become a huge challenge. This paper proposes three kinds of automatic data acquisition strategies based on (focused, incremental, custom) Web crawler technologies, which are better suited to different types of agricultural websites than traditional Web crawlers. In addition to solving asynchronous processing, dynamic page rendering, distribution, and data-persistent problems encountered during data acquisition, this paper also proposes to combine the Aho-Corasick algorithm to improve the text matching efficiency. Finally, the acquired agricultural market data was visually analyzed by using key technologies of Web mining. This study takes Chinese agricultural official websites, agricultural products wholesale market websites, and e-commerce websites as examples to integrate, process, visualize, and analyze the data acquired by using the three automatic data acquisition strategies proposed in this paper.

INTRODUCTION

Agricultural market information can objectively describe economic activities and changes of the agricultural market. It is a general name of targeted and cost-effective knowledge, news, data, intelligence that can be used for agricultural production, management, and market forecasting. From the perspective of the agricultural market demands, agricultural big data can be used to guide agricultural production, forecast agricultural market demands and support agricultural decisions, so as to achieve the desired goals of avoiding risks, increasing income, and managing transparently. Monitoring agricultural market information can ensure the balance of supply of agricultural products, effectively alert the unstable factors of the agricultural market, and ensure the sustainable and stable development of the agricultural market. With the rapid development of new-generation information technologies such as the Internet, cloud computing, and big data, various types of massive data have been rapidly formed, providing effective ways to solve the difficulties and problems faced by the development of agricultural big data. The main purpose of Web information search is to discover Web information resources by using a technology called Web crawler to automatically roam on the Internet and find target content as much as possible. Ramakrishna (2010) summed up some Web mining ideas. It makes sense to apply traditional data mining methods and Web mining ideas, to agricultural market websites and extract interesting, potential, useful patterns, and hidden information from agricultural market network resources and agricultural market network activities.

Introduction

Collaborative consumption (CC) is a new form of consumer behavior with important implications for business research (Benoit et al., 2017). CC takes place when people coordinate the acquisition and distribution of a resource. This coordination is frequently done for a fee or other non-monetary compensation through trading, bartering, and swapping (Belk, 2014). It is “the peer-to-peer-based activity of obtaining, giving, or sharing the access to goods and services, coordinated through community-based online services” (Hamari et al., 2016), p. 2047. According to some scholars, it is not clear the future growth of CC and its impact on incumbent industries (Barnes and Mattsson, 2017). However, as an emerging phenomenon from the computer-mediated interaction between customers and providers, CC is present in a vast range of business, such as transportation (Uber, Zipcar), lodging (Airbnb), tourism (Couchsurfing), entertainment (Spotify) and online food delivery services (Just-Eat.com, Clickdelivery.com, UberEATS) (Pigatto et al., 2017). Online food delivery services (OFD) offer opportunities for research as they are underrepresented in the literature of CC. According to Pigatto et al. (2017) these services can be characterized as business platforms that provide order services, payment and monitoring of the process but are not responsible for the preparation and order delivery operations. Although large fast-food chains like McDonald’s or Domino’s Pizza offer their delivery services, small or medium restaurants chains have seized the emergence of intermediaries that provide these sort of services (Yeo et al., 2017). Despite the popularity of these platforms nowadays, their connection with CC remains neglected. The words “Online Food Delivery” are missing in most recent papers of CC (Pigatto et al., 2017; Hamari et al., 2016; Benoit et al., 2017; de Rivera et al., 2017), and the publications that aim the study of OFD (Hong et al., 2016; Gupta and Paul, 2016; Yeo et al., 2017) do not mention CC either. This lack of connection does not imply that they are independent and unrelated consumption phenomena. It only means that a relevant theoretical framework that illustrates the relationship between the two remains fragmented and illdefined. Our aim here is to tackle this gap. Our approach posits the relevance of evaluating the impact of traffic conditions on the customer-provider relationship because OFD and CC are intrinsically related to urban transportation. This approach deviates from what we call the “standard methodological approach” that relies on surveys for data collection purposes. Although survey data is often intended to measure quality service through customers’ perception, this perception does not reflect delivery time accurately. We do not claim that surveys are useless to studying the customerprovider relationship. In fact, they are still present along with other data collection techniques in the researcher’s toolbox (Garrett et al., 2017; Scaraboto, 2015). Nonetheless, we grant fundamental importance to the sort of data that is available on websites or smartphone apps, as they reflect the spontaneous information of customers and providers who use a platform to enjoy the benefits of CC. The following example will illustrate this point. Imagine a client ordering a pizza, and the platform notes his pizza will be at home in less than 35 min, but it arrived an hour later due to a massive traffic jam. From a frequent customer’s view, this event might have none impact on the service perception. For a first-time customer, in contrast, this event might be perceived differently. Another important observation regarding these platforms is that they can be strategic tools for business competitiveness (Yeo et al., 2017; Jiang and Tian, 2016; Lindblom and Lindblom, 2017). For example, providers who track and adjust their delivery times according to traffic conditions might increase the number of customers, as their clients will be sure that they will receive their orderings on time or a price discount for delivery delays. This example shows that the everchanging nature of traffic dynamics is relevant for all business segments that rely on transportation to promote CC. The UberEATS application (https://www.ubereats.com/), for example, represents a successful case that shows how a brand like Uber can also be compatible with a related service like food delivery.

Abstract

View the large amount of data presented in the web pages, there is a great focus of different organizations and companies, to gather this information to use them in their best interest. This is achieved through Web mining which uses the techniques and algorithms of Data Mining to extract information and knowledge directly from the Web. The Web data mining is not only focused on the achievement of commercial information, but it is also used by various organization to make the good predictions and decisions in a different areas. The present paper deals with primary discussion of web mining, it focuses on a short presentation of web data mining its techniques, tools and applications. Then, we will propose an approach for the cross-analysis between the skills acquired in university training and the skills sought by employers in Morocco.

I. INTRODUCTION

The first use of the term Web mining (WM) goes to Oren Etzioni who tried to apply data mining technology on the Web. In this work [1], he defines Web mining as the application of data mining techniques to Web data for the extraction of relevant information from the resources available in the Web, a Web resource can be a document or a web service. Since 1996, several other works and publications have focused on this subject. The almost all of this research, it is widely accepted that Web mining is a subject of many other fields (information retrieval, databases, artificial intelligence …), which aims to extend and adapt data mining techniques [2],[3]. The data are generally stored in a Data-Wabehouse, including the objective of construction is to collect specific data to analyze the behavior of navigation we can classify the data used in four types : • Content Data: Data contained in the Web pages (texts, images, graphics…). • Data relating to the structure: structure of the page, structure inter-page… • Data relating to the Use: data providing information on the use such as IP addresses, the date and the time of queries. • Data relating to the profile of the user. In this work we will present a review of the literature on Web mining, since its techniques: last trend, its tools, and an overview on some areas of application. Then, we will propose an approach for the cross-analysis between the skills acquired in university training and the skills sought by employers in Morocco.

What is Web mining? It is the process that discover and extract the useful mode and knowledge that people are interested from the massive Web documents and activities through data mining technology [1]. Compared to the wellknown Data mining, Web mining can be extended to a deeper and broader areas, the differences between them are also very obvious: the object of data mining is the data stored in database, that is to say, the structured data; Web Mining aims at the contents or structure of Web document, which has a feature of wide-distributed, dynamic and heterogeneous, and contains unstructured or semi-structured data. Based on the diversity of information on the Web, Web mining is divided into the following category as shown in figure 1: Web structure mining, Web content mining and Web usage mining [2]. These three mining methods are different in the aspect of dealing with the main data, processing methods and application areas  1) Web Structure Mining mainly deals with Web structure data, it can be divided into page structure mining and URL mining (hyperlink mining) [3]. 2) Web Content Mining mainly deals with unstructured data and semi-structured data, can be refined into Web text mining and Web multimedia mining based on the content, in which multimedia mining is a popular research topic at present [4]. 3) Web Usage Mining can be divided into general access mode analysis and customizing Web site, it analyzes Web site logs to find some valuable knowledge. This paper will analyze the realization of Web content mining and Web structure mining, their basic algorithm principles and their application areas.

II. WEB STRUCTURAL MINING

A. Introduction of Structural Mining Massive Web sites constitute the entire Internet network, and each page in these Web sites more or less includes some hyperlinks, which contains a lot of potential information. The purpose of Web structure mining is to dig out the hidden knowledge, so that it can be fully applied. For example, if a paper is cited for a lot of times, it is proved that this paper is very authoritative in its field of study. Similarly, if there are a lot of Web pages pointing to the same page X, we think that page X has higher authority. In the field of search engine, it is very important to place the most authoritative page at the first of the search results, because when using the search engine users want to acquire authoritative and publicly recognized results, rather than some of the incorrect result pages or even insignificant ad page. Structural mining is based on this judging method to get a lot of information and help people with navigation and recommendations of the authoritative Web pages. This article will briefly introduce two well-known structure mining algorithm, PageRank and HITS. B. PageRank Algorithm The PageRank algorithm draws on the traditional citation analysis: when page A has a link to page B, we think that B gets the score that A contributes to it. This score depends on the importance of A, that is to say, the more important page A is, the higher the contribution score of page B gets. Based on this premise, Google uses this algorithm to give each page a number of values (PageRank) as a reference to the page quality. PageRank values from 0 to 10, the higher the value, the higher the quality and popularity of the page, the more forward the corresponding search results.

Introduction

Web mining is a hot topic nowadays [1–12]. Web mining algorithm can be used to find hidden information from a large number of web data, which has been widely used in e-commerce decision support applications, such as personalized recommendation. Personalized recommendation can improve e-commerce sales from three aspects: browser into buyers, increase cross selling, and build loyalty. No matter what kind of algorithm is used, the data model influences mining results greatly [13–18]. The quality of a data model can be evaluated from three parts, including collecting, storing and preparing. Data collecting can deploy in client-side, proxy-side and serve-side. Client-side collecting can be forbidden by users for privacy reasons. Proxy-side collecting can only collect the behavior of clients that send requests to the proxy. Serve-side collecting can collect all server activities of all accessing clients. Files (structured or unstructured) and databases are the main mediums for web data storing. Data preparing, including cleaning, integration, selection and transformation, is to pre-process stored data and transform them according to the requirement of web mining algorithms. At present, there are several kinds of common web data source model, including web log [19], application server log [20], Client-side log, Packet sniffer [21], and 5-gram united events model [22]. Web log is the most frequently used data source in web mining system. This data is common in a variety of web servers, such as IIS, Apache, etc. Kohavi [23] points out the shortcomings of the web log as a data source for the web mining algorithm: (1) poor data collection. First, due to the impact of local cache, proxy servers and firewalls, part of the data can’t reach the server, which causes it is not complete in web log data collection. Secondly, there are many problems in the clock synchronization of multi file system. Thirdly, the web log contains a lot of redundant information, such as the JPG, BMP and other pictures downloaded due to the need of Http protocol. (2) It is not designed for analysis. First, the URL string lacks the semantics, which is logged by web log. Especially in today’s most dynamic web site, URL analysis is almost meaningless because the page content and site structure can be dynamically generated. The second aspect is the lack of storage form information,

INTRODUCTION

WEB USAGE MINING

WEB USAGE MINING PROCESS

TECHNIQUES

APPLICATIONS

CURRENT ISSUES AND CHALLENGES

FUTURE TRENDS

CONCLUSION

Abstract

WWW is, by far, the enormous database in the world, holding humongous amount of data of various types that would be consumed by users for various needs. It contains priceless data for businesses, if mined effectively. Web mining is a process that targets to find useful information or knowledge from the Web page contents, hyperlink structure, and usage or sever logs of websites. Web data mining is divided into three major groups – Web Content mining, Web Structure mining and Web Usage mining. This survey paper reports the basic concepts of each type of web mining, techniques for data extraction or knowledge detection and uses of different types of web mining methods.

I. INTRODUCTION

World Wide Web (WWW), simply the Web, which is considered the world’s largest database, holding enormous amount of various types of data that would be consumed by us for various needs. Web data is unstructured raw information. It is a virtual gold mine for business and people, if it is mined properly. Web mining [1] is a process that targets to determine useful data from web page contents, hyperlinks organization, and server logs of websites. Web mining [2,3] processes are categorized into Web Content Mining(WCM), Web Structure Mining(WSM) and Web Usage Mining(WUM) established on the type of statistics used in the mining methods, which is given below in Fig.1. Web content mining is used to mine different web data like text, image, audio and video contents. Web structure mining is used to obtain the hyperlink organization of the web site which delivers improved search outcomes for the user. Web usage mining is a method of obtaining valuable knowledge from the webserver log files, log files of users and frequently accessed paths.

Nowadays, many people express their opinions on the Internet, including opinions about political issues. Not only governments need to know about their citizens’ opinions, but also most election campaigns and candidates are eager to know the voice of voters before the election so they can adjust their election strategy promptly. Over the years, election–related organizations have conducted polls forecasting, and the election results are mostly consistent with the prediction. However, in the Taiwan local elections held on 29 November 2014, the election results of many cities showed significant deviation from predictions. For example, the pre-election polls of the mayoral candidate of the Taoyuan city, Chih-Yang Wu, were 57 percent favorable, and those of his opponent, Wen-tsan Cheng, were only 25% favorable. However, the polls’ laggard turned out to be the winner. Evidently, simply relying on the traditional polling method doesn’t seem enough to provide an adequate forecast. With the advent of social media, such as Facebook, Plurk and Twitter, users can express their opinions anytime and anywhere. In general, young people are the main users of the Internet. As shown in Table 1, 24% of the Internet access population in Taiwan are between the ages of 15 and 24, 29% of the Internet access population are between the ages of 25 and 34. Hence, using Web reviews for election estimation mainly reflects the opinions of young people. **Insert Table 1 here** There are many poll centers in Taiwan that generally collect data using traditional methods, such as via questionnaire and interview, which are expensive and time-consuming. In recent years, there has been increasing interest in the use of data mining to translate massive amounts of social media data into useful information. The new method of using Web mining for polling is not only fast and cost-saving, but also can be used to investigate the opinions of populations that have not been reached before; it is therefore expected to replace or combine with traditional polling methods. In view of the problems faced by traditional polling, such as inadequate data for quota design and poor response rates for phone polls, this study utilized the opinions posted on social media to gauge the degree of support for two mayoral candidates of Taichung city. However, to effectively and quickly extract useful information from tons of Web data is an important issue. In particular, there are many problems in conducting opinion analysis on comments in Chinese. Hence, before performing election prediction based on Chinese social media sites, the opinion extraction rules of Chinese texts should be established first. The remainder of this paper is organized as follows. Section 2 introduces the development of social media mining in recent years with special focus on its application to election prediction. The problems of conducting semantic analysis on Chinese social media are also presented in this section. Section 3 depicts the framework of this study. The details of constructing opinion phrase extraction rules for identifying the opinion words associated with the attribute words are described. This section also introduces the six municipal governancerelated topics. Section 4 presents the results of the scores of the six topics of each candidate, based on which the election prediction is performed. Finally, Section 5 presents conclusions and future work.

With the advent of the digital age and Web 2.0, it has become essential for any business to confirm its digital presence. Consequently, the web has become “THE” channel for the companies and especially for marketers to find new prospects, improve branding, increase awareness and be closer to their consumers. Therefore, marketing strategies, in the last decade have evolved and several concepts have emerged. One thing is sure: the traditional marketing strategies are no longer holding the same place in the digital world. In this paper we define the inbound marketing strategy and web analytics best practices for improving the implementation of such a strategy in the world of Word Wide Web. The first section of the paper will take an interest in the general concepts of inbound marketing, the second part will define web mining fundamentals and the third part discuss aligning web mining and inbound marketing process .

II. LITERATURE REVIEW Traditional marketing, mass marketing or outbound marketing are different terms to name promoting a product through different media in order to propel product at the largescale and make it known. Using this process, the company is proposing to all customers its goods or services. However there are a few chances the consumer is actually being looking for this product, and therefore the rate attractiveness for such campaign can be seen in the fall and the objectives not achieved. This classic method is usually very expensive and do not offer to marketers exact ways to track the impact on people and to obtain accurate statistics on the advertising campaign especially in digital channel and web 2.0 customers.

A. Inbound Marketing The purchase process has changed, so does the selling process. Marketers need to puzzle out and innovate new ways so the customers can hear and see the offer among the increasing noise. Thus, marketers should attract customers ‘attention and make them looking for the product and being found: their sound and message must be heard through the noise .By dropping out mass advertising and email blasting, marketers must concentrate their effort on being found on the net and generating traffic to have more audience and finally building ongoing relationships with buyers.

The majority of users realize the Internet information extraction from search engines or Web browsers. These search engines do not necessarily return the information users want, both in terms of volume and in terms of content. The concept of “web mining” or “data mining of Web” can be defined as the process of discovery and analysis of useful information from the data originated. Includes three types of information: data in Internet; data “log” of Internet access servers, user registration, profiles, etc.; and web structure data. In the case of web mining content, the goal is to identify “patterns” of behavior and extract knowledge from a set of data related to documents (text, image, audio, video, etc.) stored in tables within a web environment. In the case of web data, unstructured documents with different attributes which may have similar semantics in the context of web information. The knowledge discovery “hidden” on the Internet is one of the major features of the process “web mining”. This discovered knowledge can be very useful to decision makers, helping them to identify abnormal or unknown behavior in the use or the content of the Internet [1]. Currently, the use of the term “science of data” is increasingly common, as well as the term “big data.” Here “science of data” is the study of the data knowledge extraction (heterogeneous and unstructured – texts, images and videos from networks with complex relationships between its entities). As examples, Paypal and Google use predictive models to supported business on the Internet [2]. In the context of this study, we used Google to search the set of URLs (Universal Resource Locator) and corresponding sites summaries with one or more words, particularly about the City and/or State “Rio de Janeiro” (Brazil) followed by other attributes. Depending on the research that takes place, the result can be a significant amount of URLs arranged under a “ranking” (“PageRank”, in the case of Google). This “ranking” indicates the most searched sites (quantity and quality) in a descending order according seeker’s own criteria [3], [4]. For the result of this search was as effective as possible, this study was guided then, the following research question: “How to identify patterns (or rules) in the Web mining under Multi-criteria approach?”. The choice of the Rough Set Theory (RST) and the Dominance principle (Dominance-based Rough Set Approach, DRSA) as tools to support Multi-criteria decision justified by the possibility of inaccurate data (inconsistent) and the need for treatment of these inaccuracies; and the ability to process an information system (or data table) in a mathematical perspective as well, do not need a data history as required by Fuzzy Sets (Fuzzy Set Theory) proposed by Lotfi Asker Zadeh in 1965 [5]. As support for Multi-criteria analysis, we used the jMAF software (Dominance-based Rough Set Data Analysis Framework) [6], given for purposes of research at the Computer Science Institute, Poznan University of Technology, Poland. This study includes a brief approach on Rough Set Theory (RST) and the Dominance principle (DRSA) – Sections 2 and 3, respectively; the application of the Dominance principle to a specific case, Section 4; and ends with the conclusions and directions for future studies, Section 5.

Nowadays, the increasingly competitive market has elicited an urgent need to develop successful products that can satisfy the increasing consumer expectations and demands (Cross 2000). Apart from basic functions and economic considerations, affective aspects of products are also of great concern to consumers (Jordan 1998; Khalid 2006) Design attributes, such as colour and form can provoke feelings, and influence the overall perception of a product. Therefore, affective product design (APD) was advocated to develop products that satisfy customer feelings as an aspect of product quality (Nagamachi 2002; Lai, Chang, and Chang 2005) In this regard, in the relevant literature, it is indicated that products with deliberate affective design can help improve consumer satisfaction and further promote product success (Seva, Duh, and Helander 2007) Therefore, good affective features can sharpen the competitive edge of products, and a precise understanding of product affective properties appears particularly important and deserves in-depth investigation. Regarding previous affective design studies, most research focus was placed on the analysis of emotions (Pengnate and Sarathy 2017; Gennaria et al. 2017; Huang, Chen, and Khoo 2012) and the establishment of quantitative models linking emotions and design attributes (Nagamachi 2002; Park and Han 2004; Zhai, Khoo, and Zhong 2009) In terms of data acquirement methods, surveys and user experiments are widely adopted. As in (Huang, Chen, and Khoo 2012; Huang, Chen, and Khoo 2012) questionnaires are used as the main method to collect consumer responses. However, the design of a survey, questionnaire or user experiment, to some extent, imposes the constraints of user involvement and user freedom in presenting their feelings on any design properties. Moreover, with the rapid development of the Internet techniques, Web 2.0 enables stronger interaction and participation of Internet users and leads to the participatory web (Hedges and Dunn 2018; Newman et al. 2016) Recently, the debate about Web 3.0 brings about the idea of further merging intelligent webs and web services, and correspondingly, the advancement of intelligent media and social media mining has deepened the connections between web service and users (Newman et al. 2016) New collaboration forms can be facilitated to hear customers’ voice and involve them in product design process (Djelassi and Decoopman 2013) Therefore, Web technologies can provide interactive platforms for enterprises to connect with worldwide consumers, so as to improve customer participation in the product development process and leverage open innovation. For consumers, they have more convenient channels in which to contribute their opinions. In particular, crowdsourcing, which is an important method for drawing on large numbers of people to contribute their opinions (Cross 2000; Howe 2006; Chang, Chen, and Lee 2014) has become an important way to effect consumer responses. Taking Proctor & Gamble as an example, the use of a crowdsourcing platform ‘InnoCentive’ to collect product problems and possible solutions from Internet users has helped them increase the problem solving rate to 30%. For more examples, Wikipedia, Amazon’s Mechanical Turk and iStockPhoto.com take advantage of the tremendous numbers of Web users that are willing to contribute their knowledge and ideas. Crowdsourcing appears to be a promising way to solicit consumer responses and is studied as the main data source in this work. Considering that large numbers of consumers’ comments are collected via crowdsourcing and are often presented by textual data, data mining techniques, which are efficient in dealing with big data and effective for textual analysis are considered.