Early versions of BASIC were limited to 2 character variable names. Some would allow you to use longer variable names, but only the first two characters would determine which actual variable it referenced. So if you had the variable JOHN and the variable JOAN they'd alias to the same value. Later versions of BASIC extended this to 6 characters. This was seen as positively extravagant.
Many filesystems used 8.3 filenames; you had 8 characters for the stem and 3 characters for the extension. Sometimes people say that this was just an MSDOS thing but it was common on many other OSs at the time. I'm 90% sure CP/M had this limitation too.
One aim of finite implementation limits is to define which programs are guaranteed to compile successfully, so that you don’t run into the situation that a program compiles on one implementation but not on another implementation, which would raise the question of whether it’s the program or the compiler that is nonconforming. If both are conforming, you want the program to compile 100% of the time. This isn’t possible if programs can have unlimited complexity.
The mindset is similar to embedded programming, where you have limited memory and absolutely don’t want to run into the situation that there’s not enough memory left to perform the operation that needs to be performed.