It depends on the county. Teller County, for example, exempts an agricultural building from the permit when it is used solely for agricultural uses that classify the land as agricultural, but the building must still follow the codes. See the Teller County Building Code and ask your own county early.⁠​​‍​​​⁠