Dialogue is a pervasive form of human communication. Understanding the mechanisms of language use in dialogue not only enriches our knowledge of human cognition in general, but also provides insight to understand the psychological and social aspects of communication as well, and has the potential to facilitate the design of systems, such as natural language interfaces. Language use in dialogue is characterized from multiple representation levels in most psycholinguistic theories. However, most computational evidence comes from lower level representation levels (e.g., word, syntax etc.), but little work has been done to directly model dialogue from higher levels (e.g., semantic, situational models etc.) due to technical difficulties and lack of data. To address this issue, I propose an information-theoretic perspective that views dialogue as a communication system in which information is contributed by different sources (interlocutors), and conduct a series of computational studies about higher level interaction in dialogue. First, I find evidence showing that the information contribution from interlocutors differs by their roles in initiating topic episodes, and that the aggregated information (without distinguishing interlocutor roles) conforms to the principle of maximizing communication efficiency in Information Theory. Second, I further exploit the analogy of dialogue to communication system, and view the information of utterances as time series. By applying spectral analysis techniques to those time series, I extract useful features that can effectively predict the success of collaboration in task-oriented dialogues.Third, I explore the extended implications of our proposed view by including information as a factor in analyzing the effect of social power on linguistic alignment. We find that linguistic alignment is actually influenced more by low level features such as information, rather than social power. To sum up, modeling language from an information-theoretic perspective is an effective way to study human dialogue at higher representation levels. Interaction patterns of the information representation can characterize some properties of dialogue that are not adequately captured by low level representations. Most of the low level language activities in dialogue can be accounted by the sensitivity to information content, which part of human's innate cognitive traits.