Sorting Series, which is also a kind of discrete optimization problem (i.e the permutation function
0..N-1 which maximizes
0*a[p]+1*a[p]+..+(N-1)*a[p[N-1]] is the permutation which sorts the array
a in ascending order that is
a[p] <= a[p] <= .. <= a[p[N-1]]), lies at the center of Computer Science and Algorithms because of its many uses.
Furthermore Sorting, in one way or another, is integral part of many other important algorithms and applications (see eg. Knuth TAOCP)
For example Sorting is very closely associated to Searching, another topic of immense importance and applications.
Under certain sorting states, searching can be achieved in
O(logN) time or even in
O(1) time (constant) for almost every search term.
Sorting has 3 approaches:
(eg. NIST.gov maintains a dictionary of various algorithms at: http://xlinux.nist.gov/dads// )
###Block vs. Online/Adaptive:
In the Block case, the whole array is available at once for this case many algorithms are known (comparison-based=>
O(NlogN)complexities) and (number/count based=>
O(N)complexity) (see below)
In the Adaptive/Online case, the input series is accesed one at a time (for example an time-input signal). In this case some of the previous algorithms can be transformed to work adaptively
Apart from that, there are algorithms (like Dynamic Lists, Dynamic Heaps and Balanced Trees, Tries, eg AVL Trees)
which keep an input sequence always in a 'sorted' state (with each new input) with relatively low complexity (eg
###Comparison-Based vs. Arithmetic/Count-Based:
- Comparison-based sorting algorithms (InsertionSort, MergeSort, QuickSort, etc..) sort a series by comparing elements with each other in some optimum sense
The best time complexity of these algorithms is
However better than this can be achieved
- Statistics-based sorting algorithms (CountingSort, BucketSort, RadixSort, etc..), do not use comparisons (of any kind) between elements, but instead use their arithmetic/statistical properties
This makes possible algorithms which can sort in linear
O(N) time (the fastest possible)
However these algorithms have some limitations (eg only Integers, or special kinds of Numbers)
O(N)sorting possible for arbitrary random numbers??
Computing the value of a certain number
n requires approximately
O(logn) "primitive digit" operations. Since (statisticaly) the values of numbers in a list is correlated to the size of the list itself (i.e a list of size
N contains random numbers in the range
0..N with very high probability over lists of same size for numbers in a given range), one then has an overall complexity of
O(NlogN) even for arithmetic-based sorting algorithms (see for example "what is the true complexity of radix sort?").
Classical algorithms for integer sorting require assumptions about the size of the integers to be sorted, or else have a running time dependent on the size.
However the catch here is that same holds for comparing arbitrary numbers, computationaly one has to compare
primitive digit by
primitive digit in sequence on average, hence an additional
O(logn) complexity for comparison-based algorithms.
O(NlogN)complexity a kind of strict base line for this computational model??
According to Knuth's theoretical lower bound theorem for general (comparison) sorting algorithms (note
O(logN!) = O(NlogN)): the
O(NlogN) bound is asymptoticaly tight (see also information-theoretic lower bound for comparison sorts ie Ω(NlogN) ).
A summary of various sorting/searching algorithms can be found in this pdf
- Bubble Sort
- Cocktail Sort
- Cycle Sort
- Heap Sort
- Insertion Sort
- Library Sort
- Shell Sort
- Quick Sort (recursive & iterative)
- Tree Sort
- Merge Sort (recursive, iterative & natural TODO: in-place
O(1)extra space version)
- Counting Sort
- Bucket Sort
- Radix Sort (not implemented yet)
- Intro Sort (not implemented yet)
- Burst Sort (not implemented yet)
- Tim Sort (not implemented yet)
- Permutation Sort (custom)
- Index Sort (custom)
- Statistical Sort (custom, in progress)
NOTE: The calculation of asymptotic complexity is done usually (using recursive relations) with the Master Theorem :
T(n) = aT(n/b) + f(n), a>=1, b>1
eg. for MergeSort => T(n) = 2T(n/2) + O(n) => T(n) = O(nlogn)
This package implements showcases of various (best known) sorting algorithms (and a couple of custom ones) for study, experimentation and use in applications In a concice library
Algorithms as a technology Suppose computers were infinitely fast and memory was free. Would you have any reason to study algorithms? The answer is yes, if for no other reason than that you would still like to demonstrate that your solution method terminates and does so with the correct answer. ...Of course, computers may be fast but not infinitely fast and memory may be cheap but not completely free. Computing time is therefore a bounded resource, and so is space in memory. These resources should be used wisely and algorithms that are efficient in terms of time and space will help you do so.
This demostrates that algorithms, like computer hardware, are a technology . Total system performance depends on choosing efficient algorithms as much as choosing fast hardware. Just as rapid advances are being made in other computer technologies, they are being made in algorithms as well. (Introduction to algorithms, 2nd Ed. Cormen,Leiserson,Rivest,Stein)
Algorithms as a "green" technology
Additionaly, every operation/instruction a computer performs has an energy consumption cost. So an efficient algorithm saves energy! An efficient algorithm performs a computation by trying to use the resources in the best possible manner, so effectively uses energy in the best possible manner. Where does energy come from? It comes from burning coal (mainly). So there you have it, efficient code is ecological! Better start learning your complexity soon.