this research is motivated by the need to support inference in intelligent decision support systems offered by multi-agent, distributed intelligent systems involving uncertainty. probabilistic reasoning with graphical models, known as bayesian networks (bn) or belief networks, has become an active field of research and practice in artificial intelligence, operations research, and statistics in the last two decades. at present, a bn is used primarily as a stand-alone system. in case of a large problem scope, the large network slows down inference process and is difficult to review or revise. when the problem itself is distributed, domain knowledge and evidence has to be centralized and unified before a single bn can be created for the problem. alternatively, separate bns describing related subdomains or different aspects of the same domain may be created, but it is difficult to combine them for problem solving, even if the interdependency relations are available. this issue has been investigated in several works, including most notably multiply sectioned bns (msbns) by xiang [xiang93]. msbns provide a highly modular and efficient framework for uncertain reasoning in multi-agent distributed systems. inspired by the success of bns under the centralized and single-agent paradigm, a msbn representation formalism under the distributed and multi-agent paradigm has been developed. this framework allows the distributed representation of uncertain knowledge on a large and complex environment to be embedded in multiple cooperative agents and effective, exact, and distributed probabilistic inference. what a bayesian network is, how inference can be done in a bayesian network under the single-agent paradigm, how multiple agents’ diverse knowledge on a complex environment can be structured as a set of coherent probabilistic graphical models, how these models can be transformed into graphical structures that support message passing, and how message passing can be performed to accomplish tasks in model compilation and distributed inference are covered in details in this thesis.