the next-generation of wireless networks is required to support a myriad of new demands, including higher data rate, lower latency, greater system capacity, massive connectivity, as well as increased power efficiency. among various technologies, spatial modulation (sm) and non-orthogonal multiple access (noma) have been proposed as promising technologies that can achieve high spectral and energy efficiency. furthermore, utilizing cognitive radio (cr) systems and millimeter-wave (mmwave) bandwidth have been offered as novel solutions to the spectrum scarcity problem. interestingly, sm, noma, cr, and mmwave can be integrated together to fulfill some of the upcoming generation’s wireless network requirements. being able to design of reliable transceivers that can meet these requirements is of great interest in the electronics and communications communities. direct-conversion transceivers are built with the objective of directly up-converting the baseband signal to radio frequency (rf) at the transmitter and directly down-converting the received signal at the receiver. this characteristic makes it a promising transceiver architecture due to its small size, low cost, and more efficient energy consumption. while these advantages are very favourable, the direct-conversion transceivers are susceptible to some hardware impairments (hwis) which limits the communication system performance. in-phase/quadrature-phase (i/q) imbalance represents one of the most critical direct-conversion transceivers hwis. this thesis studies the effects of i/q imbalance and improper gaussian noise (ign) on the performance of modern communication system techniques. different receiver designs are proposed to optimize the system bit error rate (ber) when the transmitter and receiver operate under the effects of the i/q imbalance in the presence of ign at the receiver. more specifically, underlay cr secondary, quadrature spatial modulation (qsm), space shift keying (ssk), and two user noma receivers are studied and analyzed. closed forms of average pairwise error probability (apep) and upper bound of the average ber formulas are derived for all receivers. these formulas are derived considering the beckmann fading channel model, where most of the well-known fading channel models can be considered as special cases. the proposed designs show solid performance against i/q imbalance effects. in fact, these effects can be totally mitigated if they exist at the receiver and can be significantly reduced if they are at the transmitter. all analytical results are verified by computer simulations. this thesis affords important results for the building of future wireless receivers. in particular, it can be used for developing digital signal processing (dsp) chips to mitigate the i/q effects and properly treat the ign at the receiver. conclusively, this thesis and the related publications can be extended to design receivers that can mitigate the effects of hwis on new promising technology. for example, the impact of hwis on cell-free massive multiple-input multiple-output (mimo) and intelligent reflecting surface (irs) is still an active issue.