Bacteria derive and maintain a variety of shapes that carry selective benefits. The shapes are usually defined by a mechanically stiff exoskeletal cell wall -- a macro-molecular network of peptidoglycan. The growth of such a network is catalyzed by transglycosylases and transpeptidases, and various cell-wall remodeling enzymes further digest and process the network. To maintain the overall cell shape, the bacterial cytoskeleton coordinates cell wall synthesis on the cellular scale. Recent studies also suggest that the mechanical properties of the bacterial cytoskeleton are important for cell wall growth. Here, we review current experiments and theories on the structure, dynamics and interactions of the bacterial cell wall and cytoskeleton, and their contributions to cell shape maintenance. We also propose future research directions that will help clarify the mystery of bacterial cell morphogenesis.