Recent advances in technology have made it possible to integrate systems with CPUs, memory units, buses, specialized logic, other digital functions and their interconnections on a single chip, giving rise to the concept of system-on-chip (SoC) architectures. In order to keep up with the incoming data rates of modern applications and to handle concurrent real-world events in real-time, multiprocessor SoC implementations have become necessary. As more processors and peripherals are being integrated on a single chip, providing an efficient and a functionally optimum interconnection has become a major design challenge. The traditional shared-bus-based approach is quite popular in SoC architectures as it is simple, well-understood and easy to implement. However, its scalability is limited, since the bus invariably becomes a bottleneck as more processors are added. Switch-based networks can overcome this bottleneck while providing true concurrency and task-level parallelism, resulting in a higher throughput. However, switch-based networks are complex and consume considerable amounts of logic resources on a chip, thus increasing the cost. Hence, the choice of switch-based networks over a bus-based architecture is an important design consideration in SoC architectures. This choice hinges on the trade-off between design simplicity and low cost vs. high communication bandwidth. This research investigates the logic resource utilization of a switch-based on-chip interconnect to analyze its scalability for multiprocessor systems. It also experimentally demonstrates the true concurrency provided by the interconnect, investigates the arbitration mechanism used, and suggests the use of a real-time operating system (RTOS) as a more effective way of managing on-chip interconnections.