Cyclic Group

The cyclic group $C_n$ with $n$ elements is the simplest kind of group. It's a commutative group which consists of all rotations of a regular $n$-gon, or all integers mod $n$ under addition, or all primitive $n$-th roots of unity under multiplication, or other equivalent isomorphic realizations (as groups). Specifically, a group is called cyclic if it is generated by a single element. In terms of permutations of a ciphertext alphabet of $n$ letters, isomorphic ciphers using this as the underlying group have alphabets consisting of all the cyclic rotations of the alphabet. These are exactly the (classical) ciphertext-autokey (CTAK) ciphers, and they can be solved using alphabet chaining without conflicts (assuming no plaintext typos).

If $n$ is prime, then every non-identity element of $C_n$ generates the group. Stated another way, if $n$ is prime, taking all multiples of any number (except for 0) mod $n$ will eventually reach all integer values mod $n$. In terms of alphabet chaining, this means that CTAK on a prime sized alphabet will have every element chain through the entire alphabet before completing a cycle.

If $n$ is not prime, then not all values will generate the group, and instead only values which are relatively prime to $n$ will. This means that when chaining, not all elements will form cycles covering the entire alphabet, and instead non-generator elements will partition the alphabet into a set of disjoint cycles of some length which is a factor of the ciphertext alphabet size.

Something relevant to chaining composite order cyclic groups which can be very useful is the Chinese Remainder Theorem (the group theory version of it). This says that if $a$ and $b$ are relatively prime, then $C_{ab} \cong C_a \times C_b$. As an example of this which is relevant to classical ciphers using an alphabet of 26 letters, $C_{26} \cong C_{13} \times C_2$. What this means is that instead of writing out an alphabet as just a line of 26 letters, you can also write it as two rows of 13 letters and do 2-dimensional chaining, with an order 2 element swapping the two rows. This might seem kind of obvious because $26 = 13 \times 2$, but this actually doesn't work when the values aren't relatively prime. For example, if you have an alphabet size of 25, there are two completely different group structures, $C_{25}$ and $C_5 \times C_5$, and 2-dimensional chaining with a 5x5 alphabet doesn't work for $C_{25}$ (though you can certainly still lay out the letters that way). Most products of more than one cyclic group will not be cyclic (meaning they cannot be generated by one element), the Chinese remainder theorem just gives the condition for when a cyclic group will have an equivalent polycyclic representation, or when polycyclic groups can be reduced.

Ciphers using polycyclic state groups (commutative group ciphertext-autokey (GCTAK) ciphers) cannot be solved with classical alphabet chaining, but they can be solved with multi-dimensional alphabet chaining, which is when you consider the transformations between isomorphs as vectors in some number of dimensions, instead of just distances in one dimension.