In the presence of less labeled data for training a classifier, active learning can be applied to interactively query the user about the label of an input. Active learning has been the subject of intensive investigation over the last 5 decade Zhu et al. (2007). However, most of the work has focussed on offline active learning Nguyen & Smeulders (2004); Cohn et al. (1996). Indeed, few attempts have been made to develop online active learning algorithms for data streams Ienco et al. (2013); Attenberg & Provost (2011). By its very nature, active learning and in particular stream-based active 10 learning is relevant to various applications where external feedback from the environment is used to enhance the classification performance. Learning from social media (SM) data for a particular application like crisis management may require active involvement of the users who could be emergency management staff members (e.g. first responders) to label ambiguous items. Interestingly, in 15 recent years, SM has become a well-established communication medium for the public to exchange information. Mobile devices and mobile Internet allow users to publish information almost anywhere at anytime. This makes SM a very important source of information for different purposes. There is considerable effort by the research community to harness social media for emergency man20 agement. Many studies in the context of SM and emergency management show the usefulness of this data for emergency preparation, response planning, and recovery strategies Pohl et al. (2013); Pohl (2014). More precisely, emergency departments have already noticed the importance of social media for gathering, monitoring, and disseminating information Denef et al. (2013); Hughes et al. 25 (2014) but they mostly use simple built-in browsing and search mechanisms as for Twitter. To exploit SM in the context of crisis management, it is necessary to identify data that is relevant to the crisis/emergency at hand. Hence, appropriate preprocessing is necessary in order to distinguish between relevant and irrelevant 30 items by involving the emergency management staff. Such professionals can then share their experience and knowledge to develop useful learning systems. In this paper, we propose a batch-based active learning algorithm (OBAL). This algorithm uses label uncertainty Cohn et al. (1994) as query criterion. It does that by examining the boundary separating the classes, which represent 35 uncertainty regions. Here, the classes are ”relevant” and ”not relevant”. OBAL self-adapts in response to changes in the data stream (i.e., shifts in the boundary region) by continuously updating the boundary information.