Think about observations and actions as about points in space. E.g., if the type of
Dicrete(16) like in frozen lake, each observation is an integer from 0 to 15 and
observation_space.n is how you get the value in the parentheses (16 in this case).
In cart-pole the type of the observation space is
Box(4,), thus, its bounds are defined by
observation_space.low instead of
observation_space.n. You can find the details in the section on spaces of the documentation.