Smaller carpenter ants typically live in decaying wood, such as rotting tree stumps or old wooden structures, where they create galleries and nests.