tag:blogger.com,1999:blog-84014989875555821122024-02-08T02:12:08.779+00:00cat /dev/thoughts~$ cat /dev/thoughts > $BLOG # Programming things, geek things, random things...JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.comBlogger16125tag:blogger.com,1999:blog-8401498987555582112.post-89487043250202985802013-07-08T23:08:00.003+01:002013-07-08T23:08:56.400+01:00What Every Computer Scientist Should Know About Random NumbersThis article outlines a few key things you should know about random numbers if you're a programmer, in the same vein as <a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.6768">What Every Computer Scientist Should Know About Floating Point Arithmetic</a> by David Goldberg (a more accessible version of which is available at <a href="http://floating-point-gui.de/">The Floating-Point Guide</a>) and Joel Spolsky's <a href="http://www.joelonsoftware.com/articles/Unicode.html">The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets</a>.
<br />
<br />
But this will be much shorter than those articles.
<br />
<br />
<h2>The Basics</h2>
<br />
<h3>What are random numbers?</h3>
<br />
A <b>random number sequence</b> is one where the next number in the sequence cannot be predicted, even if you know all of the previous numbers—i.e. one that contains no discernible pattern.
<br />
<br />
A <b>random number generator</b> is a device or program that produces such a pattern.
<br />
<br />
<h3>Why do we need random numbers?</h3>
<br />
They have applications in computer games, simulations and computer security. Simple examples would be picking the next track for a music player to select in shuffle mode or the order of a deck of cards in an online poker game. Additionally, many cryptography and security applications rely on starting with some unknown (and unpredictable) number or sequence of numbers, and their security falls down if they can be predicted.
<br />
<br />
<h3>How can a computer, which is deterministic, produce random numbers?</h3>
<br />
Unless you take special measures (see the next section) computers can't produce <i>truly</i> random numbers, so instead they do something pretty close that's usually good enough—they produce <b>pseudo-random numbers</b>, which are sequences of numbers that (i) appear random to humans and (ii) are <i>statistically random</i>. Such sequences generally have a uniform distribution—i.e. all numbers within the range of numbers produced by the random number generator are equally likely to be chosen. Another way of looking at this is that if you generate many pseudo-random numbers, each number within the range will be chosen roughly the same number of times, so no numbers will be preferred or chosen more frequently than any others. Of course there are other distributions you may want (e.g. a normal/Gaussian distribution), but usually you'll have a good reason for this (if you've never heard of normal or Gaussian distributions, they're very unlikely to be what you want).
<br />
<br />
Pseudo-random number sequences generally start with a <b>seed value</b> from which all the other numbers in the sequence follow using some algorithm. The algorithm itself is deterministic, and this is what makes the numbers it generates <i>pseudo</i> random as opposed to truly random.
<br />
<br />
<h3>What about if I want <i>truly</i> random numbers?</h3>
<br />
This is generally done using hardware entropy—the operating system detects things about hardware which cannot be predicted, such as when traffic arrives on your network port, and uses it to generate numbers that are truly random. On most <tt>UNIX</tt> systems these numbers can be accessed through <tt>/dev/random</tt>, which simply produces random bytes whenever you read from it. However, this method is a <i>lot</i> slower than producing pseudo-random numbers (just try running <tt>hd /dev/random</tt> on any <tt>UNIX</tt> system; wiggle the mouse or pound the keyboard to hurry things up), and doesn't have the advantage of being able to repeat the sequence for debugging/testing.
<br />
<br />
<h2>Random Number APIs</h2>
<br />
Most languages' standard libraries come with facilities to generate random numbers. They generally consist of at least two basic things:
<ol>
<li>A way to seed a sequence of pseudo-random numbers, and</li>
<li>A way to get the next number in the sequence.</li>
</ol>
Sometimes it's useful to have the same seed, for example for reproducing problems when debugging. Other times it's necessary to choose a seed from a source such as the current time in order to get a different pseudo-random sequence on each run of a program. Be careful choosing your seed from the time—depending on the resolution of the timer you're using, you could end-up with the same sequence being used on consecutive runs of a program (see <a href="http://catdevthoughts.blogspot.co.uk/2012/09/seeding-randomness.html">an earlier post</a> about this).
<br />
<br />
<h2>Common Mistakes when Producing Random Numbers</h2>
<br />
These are the mistakes most beginners make, especially when asking for help on forums.
<br />
<br />
<h3>Trying to make the numbers “more random”</h3>
<br />
It is a common mistake for people to assume that “pseudo-random” really means “not-very-random” and that they should try to somehow “improve” this randomness. This is typically done by calling some random number generation function (e.g. the standard C <tt>rand()</tt> or Java's <tt>Math.random()</tt>) several times and combining the results, as seen in <a href="http://thedailywtf.com/Articles/Classic-WTF-SuperRand.aspx">this post</a> on The Daily WTF, for example.
<br />
<br />
Producing random numbers with a uniform statistical distribution is very hard—you need some pretty advanced number theory to really know what you're doing and get it right. In many cases, combining several random numbers in this way produces a bias for/against some numbers, which actually makes these functions <i>less</i> random than they started out.
<br />
<br />
On the other hand if you do think you've produced better random numbers than some other implementation, <a href="http://csrc.nist.gov/groups/ST/toolkit/rng/index.html">NIST</a> has some statistical tests you can download to check this.
<br />
<br />
<h3>Re-seeding the random number generator every time you need a new random number</h3>
<br />
This is another common mistake—a pseudo-random number sequence should generally be seeded once and only once. You'll only need to re-seed such a sequence if you want to reproduce a previous run of random numbers (e.g. for debugging/testing). Frequently when people attempt to re-seed the generator they make use of the current time, which can lead to the same “random” number being used several times in a row if the timer resolution isn't fine enough and the random number function is called often enough.
<br />
<br />
<h2>How You Should Produce Random Numbers...</h2>
<br />
How you should produce random numbers depends mostly on what's at stake—i.e. what the consequences would be if someone were able to <i>predict</i> the next random number in a sequence.
<br />
<br />
<h3>...when no personal data is at stake</h3>
<br />
This will be the case for most random number applications, e.g. for games that need to shuffle cards/simulate dice rolls, etc.
<br />
<br />
In this case, use the random number generator that comes with your language's standard library, seeded with the current time. If you need the random number to be within a certain range, take the modulus and add the base of the range, e.g. in C:
<br />
<br />
<pre class="brush:c">
/*
* Get a random number in range 100 to 110, inclusive.
* There are 11 numbers to choose from (110 - 100 + 1 = 11).
*/
int x = (rand() % 11) + 100;
</pre>
<br />
This will work fine so long as the low-order bits of your implementation are as random as the high-order bits, which they will be in any half-way decent implementation. If you're worried about the quality of your standard library's implementation, check out the <a href="http://www.gnu.org/software/gsl/manual/html_node/Random-Number-Generation.html">GNU Scientific Library</a>'s random number facilities.
<br />
<br />
If your timer resolution is in seconds and you're worried about people getting the same seed by running two instances of your program in quick succession, combine the current time with the process ID. Alternatively on <tt>UNIX</tt> you can use <tt>/dev/urandom</tt> to generate either a seed or a stream of pseudo-random numbers.
<br />
<br />
You may also want to consider (i) printing out the seed you use and (ii) adding a flag to set the seed to an explicit value. This will allow you to reproduce problems that may only occur when a certain sequence of numbers is generated.
<br />
<br />
<h3>...when personal data <i>is</i> at stake</h3>
<br />
This will be the case for cryptographic applications where bank details, passwords and the like are being protected. These normally involve a small quantity of random numbers being used once to generate a key, so performance is rarely an issue.
<br />
<br />
In this case, using a true random number generator is essential. On most <tt>UNIX</tt> systems you can use <tt>/dev/random</tt> for this purpose, though if you need a large quantity of random numbers this may take some time—encourage the user to wiggle the mouse in order to generate more hardware entropy (seriously).
<br />
<br />
If your platform doesn't have such a facility, <a href="https://www.random.org/">random.org</a> can produce truly random <a href="https://www.random.org/bytes/">bytes</a> over a secure HTTPS connection. And if this isn't an option and a true random number generator isn't available, <tt>/dev/urandom</tt> is probably the safest fallback.
<br />
<br />
<h2>References</h2>
<br />
<ul>
<li>
<a href="http://csrc.nist.gov/groups/ST/toolkit/rng/index.html">http://csrc.nist.gov/groups/ST/toolkit/rng/index.html</a>
<br />
NIST's statistical tests for random number generators.
</li>
<li>
<a href="http://www.gnu.org/software/gsl/manual/html_node/Random-Number-Generation.html">http://www.gnu.org/software/gsl/manual/html_node/Random-Number-Generation.html</a>
<br />
Documentation for the GNU Scientific Library's random number generation facilities.
</li>
<li>
<a href="https://www.random.org/">https://www.random.org/</a>
<br />
Random.org—both free and paid true-random-number generation facilities.
</li>
<li>
<a href="http://linux.die.net/man/4/random">http://linux.die.net/man/4/random</a>
<br />
Online man page for <tt>/dev/random</tt> and <tt>/dev/urandom</tt>.
</li>
</ul>
JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-7419726723348245932013-04-04T22:15:00.000+01:002013-04-04T22:19:19.473+01:00How to Get the Programming Job You WantI've been on a job hunt recently, and have just been offered a position that is pretty much the ideal position I was after. In light of this, I thought I'd share a quick piece of advice on how to get the programming job you really want.
<br />
<br />
While I believe this advice certainly applies to programming, it may apply to other fields as well, especially where there's a low barrier to entry. Also, note that this advice even applies if you've never had a programming job before, and if you have absolutely no formal qualifications in programming (I don't, and I had to start somewhere…).
<br />
<br />
My advice is a fairly simple, two-stage process:
<ol>
<li>Figure out what programming job you want, and</li>
<li>Start doing it.</li>
</ol>
As I mentioned before, I have no formal programming qualifications—but for entry-level positions most employers are willing to look past this, so long as you can show that you have the ability to actually <i>do the job</i>. And what better way to show you can do the job by doing the job in your spare time? Additionally, since software development is a profession where you're constantly learning, being able to demonstrate that you're an “information sponge” is a big plus. Sure, the guy just out of university knows as much Java as you do, but did he just do it to pass his assignments? If you haven't had assignments to pass, you must have learnt it because (i) you were eager to and (ii) you were able to—you're proactive, not just another <a href="http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html">JavaSchool</a> drone. This logic applies whether you're new to software development, or just to some specific area of software development.
<br />
<br />
With easy access to open-source and free-of-charge software and relatively cheap hardware, there's a plethora of ways you can get real-world, genuinely valuable experience in your field that'll give you something to put on your CV* and show off at interview to demonstrate you know what you're talking about. You want to write .NET desktop apps? Go write some—re-implement Task Manager if you want, it doesn't matter if it's been written before. Want to work on operating system kernels? Linux and the various BSD variants will accept contributions from anyone, or you could <a href="wiki.osdev.org/">write your own</a>. Want to write websites? Find an excuse to make one, even if it's just a personal one to show-off your LAMP/AJAX skills. Want to program safety-critical real-time embedded systems? Well, you probably can't do that recreationally—but you can get fairly cheap evaluation boards from a variety of sources, get a book on real-time systems and put the two together.
<br />
<br />
Whatever you want to do, just go ahead and do it—not only will employers relish in the fact that you've given yourself your own on-the-job training, but you'll find out whether you really like that area.
<br />
<br />
Of course, <b>there's a flip-side</b>—at the end of it you <i>have to know your stuff</i>. You won't have the presumed knowledge that comes with a formal qualification, so be prepared to demonstrate the limits of your knowledge when you interview. Make sure you study as much as possible in your chosen area—read (good) books, visit online forums to try to answer other people's questions (especially if you have to research the answer) and learn everything you can about your chosen language/technologies (they're the tools of the trade, after all).
<br />
<br />
<br />
* <i>That's a résumé, for you Americans.</i>
JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-51302789035692116452012-10-18T21:42:00.002+01:002013-04-15T22:56:57.895+01:00Pthreads OverviewHere's a quick overview of the calls available to a programmer using Pthreads (both the standard Pthreads API and GNU extensions). It's intended to be as brief as possible while being (i) descriptive and (ii) not just a list of all the available calls. The links are to the relevant man pages on <a href="http://linux.die.net/">linux.die.net</a>.
<br />
<br />
<i>Note: The "NP" at the end of certain functions and macros stands for "non-portable". They are GNU extensions, accessed by e.g. <tt>#define</tt>-ing <tt>_GNU_SOURCE</tt>.</i>
<br />
<br />
<ul>
<li><a href="#creation">Thread creation and attributes</a></li>
<li><a href="#ops">Thread operations</a></li>
<ul>
<li><a href="#termhandle">Fork/thread termination handlers</a></li>
<li><a href="#termination">Thread termination and cancellation</a></li>
<li><a href="#signals">Signal operations</a></li>
<li><a href="#sched">Scheduling and CPU time/affinity</a></li>
</ul>
<li><a href="#specific">Thread-specific data and one-time-only operations</a></li>
<li><a href="#locking">Locking mechanisms</a></li>
<ul>
<li><a href="#mutex">Mutexes</a></li>
<li><a href="#rwlock">Read/write locks</a></li>
<li><a href="#spin">Spin locks</a></li>
</ul>
<li><a href="#barrier">Memory barriers</a></li>
<li><a href="#cond">Condition variables</a></li>
</ul>
</br>
<h2><a name=creation>Thread creation and attributes</a></h2>
<br />
A thread is created with <tt><a href="http://linux.die.net/man/3/pthread_create">pthread_create()</a></tt>. Thread attributes can be specified when it is created with a <tt>pthread_attr_t</tt> structure, which gets initialized with <tt><a href="http://linux.die.net/man/3/pthread_attr_init">pthread_attr_init()</a></tt> and destroyed with <tt><a href="http://linux.die.net/man/3/pthread_attr_destroy">pthread_attr_destroy()</a></tt>. A copy of a thread's current attributes can be obtained with the GNU-specific <tt><a href="http://linux.die.net/man/3/pthread_getattr_np">pthread_getattr_np()</a></tt>.
<br />
<br />
The <tt>pthread_attr_t</tt> can have the following operations performed on it:
<ul>
<li>The thread's initial CPU affinity mask can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getaffinity_np">pthread_attr_getaffinity_np()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setaffinity_np">pthread_attr_setaffinity_np()</a></tt>.</li>
<li>Whether the thread is detached or joinable can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getdetachstate">pthread_attr_getdetachstate()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setdetachstate">pthread_attr_setdetachstate()</a></tt>.</li>
<li>The size of the guard area (area at the end of the stack that will generate a segmentation fault on access, to detect stack overflow) can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getguardsize">pthread_attr_getguardsize()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setguardsize">pthread_attr_setguardsize()</a></tt> (but note that guard sizes are rounded up to page boundaries when applied).</li>
<li>Whether the thread gets its scheduling policy and parameters (see the next two points) by inheriting them from the current thread or explicitly from the values set in the attribute structure can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getinheritsched">pthread_attr_getinheritsched()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setinheritsched">pthread_attr_setinheritsched()</a></tt>.</li>
<li>The scheduling policy the new thread starts with (assuming it's been set to have this explicitly specified; see above) can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getschedpolicy">pthread_attr_getschedpolicy()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setschedpolicy">pthread_attr_setschedpolicy()</a></tt>.</li>
<li>The scheduling parameters (i.e. priority, unless more parameters are added in future) the new thread starts with (assuming it's been set to have this explicitly specified; see above) can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getschedparam">pthread_attr_getschedparam()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setschedparam">pthread_attr_setschedparam()</a></tt>.</li>
<li><i>Note: Only system-level contention is available on Linux.</i> Whether the scheduling contention scope of the thread is at system-level (i.e. the thread's priority is compared with every other thread on the system when allocating CPU time) or process-level (i.e. the thread's priority is only compared with other threads in the process; the process itself must get CPU time before the scheduler decides if the thread deserves CPU time) can be accessed/modified by <tt><a href="http://linux.die.net/man/3/pthread_attr_getscope">pthread_attr_getscope()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setscope">pthread_attr_setscope()</a></tt>.</li>
<li>If a thread needs to run with a stack with a specific location and size, this can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getstack">pthread_attr_getstack()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setstack">pthread_attr_setstack()</a></tt>. The size alone can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_attr_getstacksize">pthread_attr_getstacksize()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setstacksize">pthread_attr_setstacksize()</a></tt>. <i>(The functions for accessing and modifying the address are <tt><a href="http://linux.die.net/man/3/pthread_attr_getstackaddr">pthread_attr_getstackaddr()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_attr_setstackaddr">pthread_attr_setstackaddr()</a></tt>, but they are obsolete)</i>.</li>
</ul>
<br />
<h2><a name=ops>Thread operations</a></h2>
<br />
A thread can get its own ID via <tt><a href="http://linux.die.net/man/3/pthread_self">pthread_self()</a></tt>. Two thread IDs can be tested to see if they refer to the same thread with <tt><a href="http://linux.die.net/man/3/pthread_equal">pthread_equal()</a></tt> (<i>don't</i> just use <tt>==</tt>).
<br />
<br />
<h3><a name=termhandle>Fork/thread termination handlers</a></h3>
<br />
You can specify functions to be executed immediately before and after a call to <tt>fork()</tt> (in both parent and child) with <tt><a href="http://linux.die.net/man/3/pthread_atfork">pthread_atfork()</a></tt>. This is useful in e.g. a threaded library, since the child process a <tt>fork()</tt> creates only consists of a single thread.
<br />
<br />
Thread clean-up handlers (analogous to <tt>atexit()</tt>/<tt>on_exit()</tt>) can be registered/unregistered with <tt><a href="http://linux.die.net/man/3/pthread_cleanup_push">pthread_cleanup_push()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_cleanup_pop">pthread_cleanup_pop()</a></tt>. They will be called if the thread is cancelled or it exits using <tt><a href="http://linux.die.net/man/3/pthread_exit">pthread_exit()</a></tt> (but not if it just returns from the thread routine).
<br />
<br />
If you're worried about asynchronous cancellation between pushing/popping clean-up handlers (and who isn't!) you can use the GNU-specific <tt><a href="http://linux.die.net/man/3/pthread_cleanup_push_defer_np">pthread_cleanup_push_defer_np()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_cleanup_pop_restore_np">pthread_cleanup_pop_restore_np()</a></tt> instead, which (i) set the thread cancellation type to deferred (see <a href="#termination">Thread termination</a>) and (ii) restore the previous thread cancellation type, respectively.
<br />
<br />
<h3><a name=termination>Thread termination and cancellation</a></h3>
<br />
A thread can exit from a thread at any time using <tt><a href="http://linux.die.net/man/3/pthread_exit">pthread_exit()</a></tt>.
<br />
<br />
A thread that is joinable can be joined with <tt><a href="http://linux.die.net/man/3/pthread_join">pthread_join()</a></tt>, which will block until the thread has finished. A thread can be moved to the detached state with <tt><a href="http://linux.die.net/man/3/pthread_detach">pthread_detach()</a></tt> (the reverse cannot be done). Normally, an attempt to join will block until the thread has finished, but this blocking behaviour can be avoided with the GNU-specific <tt><a href="http://linux.die.net/man/3/pthread_tryjoin_np">pthread_tryjoin_np()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_timedjoin_np">pthread_timedjoin_np()</a></tt>.
<br />
<br />
One primitive way of terminating a thread is to cancel it with <tt><a href="http://linux.die.net/man/3/pthread_cancel">pthread_cancel()</a></tt>. Whether the thread is able to be cancelled in this way can be controlled with <tt><a href="http://linux.die.net/man/3/pthread_setcancelstate">pthread_setcancelstate()</a></tt>, and whether such a cancellation is deferred or delivered at a cancellation point can be controlled with <tt><a href="http://linux.die.net/man/3/pthread_setcanceltype">pthread_setcanceltype()</a></tt>. Within a thread, an "artificial" cancellation point can be generated by calling <tt><a href="http://linux.die.net/man/3/pthread_testcancel">pthread_testcancel()</a></tt>.
<br />
<br />
<h3><a name=signals>Signal operations</a></h3>
<br />
A signal can be sent to a specific thread using <tt><a href="http://linux.die.net/man/3/pthread_kill">pthread_kill()</a></tt> or <tt><a href="http://linux.die.net/man/3/pthread_sigqueue">pthread_sigqueue()</a></tt>, analogous to the normal <tt>kill()</tt> and <tt>sigqueue</tt> functions.
<br />
<br />
If you're using LinuxThreads (you'll probably know if you are; most systems are using the Native POSIX Thread Library now, for which this function is irrelevant) you can use <tt><a href="http://linux.die.net/man/3/pthread_kill_other_threads_np">pthread_kill_other_threads_np()</a></tt> to send <tt>SIGKILL</tt> to all other threads, usually to correct a deficiency whereby other threads are not terminated when calling one of the <tt>exec()</tt> family of functions.
<br />
<br />
The signal mask for a thread can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_sigmask">pthread_sigmask()</a></tt>, which uses a <tt>sigset_t</tt> that can be manipulated with the usual <tt><a href="http://linux.die.net/man/3/sigsetops">sigsetops</a></tt>.
<br />
<br />
<h3><a name=sched>Scheduling and CPU time/affinity</a></h3>
<br />
The <tt>clock_t</tt> of a specific thread (e.g. for a call to <tt><a href="http://linux.die.net/man/2/clock_gettime">clock_gettime()</a></tt>) can be accessed with <tt><a href="http://linux.die.net/man/3/pthread_getcpuclockid">pthread_getcpuclockid()</a></tt>.
<br />
<br />
A thread's scheduling policy and parameters (i.e. priority) can be accessed/modified using <tt><a href="http://linux.die.net/man/3/pthread_setschedparam">pthread_setschedparam()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_getschedparam">pthread_getschedparam()</a></tt>. The priority alone can be set using <tt><a href="http://linux.die.net/man/3/pthread_setschedprio">pthread_setschedprio()</a></tt>. The CPU can be released with <tt><a href="http://linux.die.net/man/3/pthread_yield">pthread_yield()</a></tt>, which is analogous to <tt><a href="http://linux.die.net/man/2/sched_yield">sched_yield()</a></tt> but for an individual thread.
<br />
<br />
<i>Note: This is irrelevant on Linux, and is ignored.</i> The concurrency (a hint to the system about how many lower-level entities (e.g. kernel threads) to assign) of a process can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_getconcurrency">pthread_getconcurrency()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_setconcurrency">pthread_setconcurrency()</a></tt>.
<br />
<br />
The CPU affinity of a thread can be accessed/modified after creation with the GNU-specific <tt><a href="http://linux.die.net/man/3/pthread_getaffinity_np">pthread_getaffinity_np()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_setaffinity_np">pthread_setaffinity_np()</a></tt>.
<br />
<br />
<h2><a name=specific>Thread-specific data and one-time-only operations</a></h2>
<br />
Once-only initialization can be achieved using <tt><a href="http://linux.die.net/man/3/pthread_once">pthread_once()</a></tt>. This requires a <tt>pthread_once_t</tt> which is initialized by assignment from the <tt>PTHREAD_ONCE_INIT</tt> macro.
<br />
<br />
Thread-local data is provided through a <tt>pthread_key_t</tt>, which is initialized/freed using <tt><a href="http://linux.die.net/man/3/pthread_key_create">pthread_key_create()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_key_delete">pthread_key_delete()</a></tt>. The value can then be accessed/modified using <tt><a href="http://linux.die.net/man/3/pthread_getspecific">pthread_getspecific()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_setspecific">pthread_setspecific()</a></tt>.
<br />
<br />
<br />
<h2><a name=locking>Locking mechanisms</a></h2>
<br />
Pthreads provides three types of locking mechanism - mutexes, read/write locks and spin locks.
<br />
<br />
<h3><a name=mutex>Mutexes</a></h3>
<br />
These are the most common types of lock, as well as the most flexible. They are also the only locks that can be used with condition variables.
<br />
<br />
A mutex is initialized/freed with <tt><a href="http://linux.die.net/man/3/pthread_mutex_init">pthread_mutex_init()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_mutex_destroy">pthread_mutex_destroy()</a></tt>. You can also initialize a statically allocated mutex using the <tt><a href="http://linux.die.net/man/3/pthread_mutex_init">PTHREAD_MUTEX_INITIALIZER</a></tt> macro or, as GNU extensions, <tt>PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP</tt>, <tt>PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP</tt> or <tt>PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP</tt>.
<br />
<br />
Mutexes can optionally be initialized with attributes in the form of a <tt>pthread_mutexattr_t</tt>, initialized and freed using <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_init">pthread_mutexattr_init()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_destroy">pthread_mutexattr_destroy()</a></tt>.
The <tt>pthread_mutexattr_t</tt> can have the following operations performed on it:
<ul>
<li>The type of mutex used can be accessed/modified using <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_gettype">pthread_mutexattr_gettype()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_settype">pthread_mutexattr_settype()</a></tt>. A mutex can be normal, error-checking (re-locking the mutex from the owning thread is an error) or recursive (relocking the mutex from the owning thread is fine).</li>
<li>Whether a mutex can be used from multiple processes can be accessed/modified using <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_getpshared">pthread_mutexattr_getpshared()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_setpshared">pthread_mutexattr_setpshared()</a></tt>.</li>
<li>To prevent priority inversion, the behaviour of a thread's scheduling while a mutex is locked and/or blocking higher priority threads can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_getprotocol">pthread_mutexattr_getprotocol()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_setprotocol">pthread_mutexattr_setprotocol()</a></tt>. The priority ceiling of a mutex can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_getprioceiling">pthread_mutexattr_getprioceiling()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_mutexattr_setprioceiling">pthread_mutexattr_setprioceiling()</a></tt>.</li>
</ul>
<br />
The priority ceiling of a thread can be accessed/modified after creation using <tt><a href="http://linux.die.net/man/3/pthread_mutex_getprioceiling">pthread_mutex_getprioceiling()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_mutex_setprioceiling">pthread_mutex_setprioceiling()</a></tt>.
Mutexes can be locked using any of <tt><a href="http://linux.die.net/man/3/pthread_mutex_lock">pthread_mutex_lock()</a></tt>, <tt><a href="http://linux.die.net/man/3/pthread_mutex_timedlock">pthread_mutex_timedlock()</a></tt> or <tt><a href="http://linux.die.net/man/3/pthread_mutex_trylock">pthread_mutex_trylock()</a></tt>. They can be unlocked with <tt><a href="http://linux.die.net/man/3/pthread_mutex_unlock">pthread_mutex_unlock()</a></tt>.
<br />
<br />
<h3><a name=rwlock>Read/write locks</a></h3>
<br />
Read/write locks can provide performance benefits over mutexes since they allow any number of threads to access the protected resource with "read-only" behaviour, while granting exclusive access to threads that want to be able to modify the resource. They have some disadvantages over mutexes though, namely that they can't be configured to avoid priority inversion and they can't be used with condition variables.
<br />
<br />
A read/write lock is initialized/freed with <tt><a href="http://linux.die.net/man/3/pthread_rwlock_init">pthread_rwlock_init()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_rwlock_destroy">pthread_rwlock_destroy()</a></tt>. You can initialize a statically allocated read/write lock with <tt><a href="http://linux.die.net/man/3/pthread_rwlock_init">PTHREAD_RWLOCK_INITIALIZER</a></tt> or, as a GNU extension, <tt>PTHREAD_RWLOCK_WRITER_NONRECURSIVE_INITIALIZER_NP</tt>.
<br />
<br />
Read/write locks can optionally be initialized with attributes in the form of a <tt>pthread_rwlockattr_t</tt>, initialized and freed using <tt><a href="http://linux.die.net/man/3/pthread_rwlockattr_init">pthread_rwlockattr_init()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_rwlockattr_destroy">pthread_rwlockattr_destroy()</a></tt>.
<br />
<br />
Currently, the only attribute a read/write lock can be given is whether it can be used by multiple processes. This setting can be accessed/modified with <tt><a href="http://linux.die.net/man/3/pthread_rwlockattr_getpshared">pthread_rwlockattr_getpshared()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_rwlockattr_setpshared">pthread_rwlockattr_setpshared()</a></tt>.
<br />
<br />
A read lock can be obtained using one of <tt><a href="http://linux.die.net/man/3/pthread_rwlock_rdlock">pthread_rwlock_rdlock()</a></tt>, <tt><a href="http://linux.die.net/man/3/pthread_rwlock_timedrdlock">pthread_rwlock_timedrdlock()</a></tt> or <tt><a href="http://linux.die.net/man/3/pthread_rwlock_tryrdlock">pthread_rwlock_tryrdlock()</a></tt>. The corresponding functions for obtaining the write lock are <tt><a href="http://linux.die.net/man/3/pthread_rwlock_timedwrlock">pthread_rwlock_timedwrlock()</a></tt>, <tt><a href="http://linux.die.net/man/3/pthread_rwlock_trywrlock">pthread_rwlock_trywrlock()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_rwlock_wrlock">pthread_rwlock_wrlock()</a></tt>. The lock can be unlocked from either type of lock with <tt><a href="http://linux.die.net/man/3/pthread_rwlock_unlock">pthread_rwlock_unlock()</a></tt>.
<br />
<br />
<h3><a name=spin>Spin locks</a></h3>
<br />
These are the simplest form of locks, and are generally only used when you know that other threads will only be holding the locks for a short amount of time (e.g. not whilst performing blocking I/O).
<br />
<br />
A spin lock is initialized/freed with <tt><a href="http://linux.die.net/man/3/pthread_spin_init">pthread_spin_init()</a></tt> and<tt><a href="http://linux.die.net/man/3/pthread_spin_destroy">pthread_spin_destroy()</a></tt>. There is no associated attributes structure (but they can be made available to other processes upon initialization).
<br />
<br />
The functions for locking/unlocking a spin lock are <tt><a href="http://linux.die.net/man/3/pthread_spin_lock">pthread_spin_lock()</a></tt>, <tt><a href="http://linux.die.net/man/3/pthread_spin_trylock">pthread_spin_trylock()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_spin_unlock">pthread_spin_unlock()</a></tt>.
<br />
<br />
<br />
<h2><a name=barrier>Memory barriers</a></h2>
<br />
These allow threads to synchronize at a pre-defined point of execution.
<br />
<br />
They are initialized and freed using <tt><a href="http://linux.die.net/man/3/pthread_barrier_init">pthread_barrier_init()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_barrier_destroy">pthread_barrier_destroy()</a></tt>.
<br />
<br />
A memory barrier can optionally be initialized with attributes in the form of a <tt>pthread_barrierattr_t</tt>, initialized and freed using <tt><a href="http://linux.die.net/man/3/pthread_barrierattr_init">pthread_barrierattr_init()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_barrierattr_destroy">pthread_barrierattr_destroy()</a></tt>.
<br />
<br />
Currently, the only attribute a memory barrier can be given is whether it can be used by multiple processes. This property can be accessed/modified using <tt><a href="http://linux.die.net/man/3/pthread_barrierattr_getpshared">pthread_barrierattr_getpshared()</a></tt> and<tt><a href="http://linux.die.net/man/3/pthread_barrierattr_setpshared">pthread_barrierattr_setpshared()</a></tt>.
<br />
<br />
Once created, threads can synchronize at a barrier using <tt><a href="http://linux.die.net/man/3/pthread_barrier_wait">pthread_barrier_wait()</a></tt>. There is no such thing as a "trywait" or "timedwait" function.
<br />
<br />
<br />
<h2><a name=cond>Condition variables</a></h2>
<br />
Condition variables can be used to signal changes in an application's state to other threads. They must be used with an associated <tt>pthread_mutex_t</tt>.
<br />
<br />
Condition variables are initialized/freed using <tt><a href="http://linux.die.net/man/3/pthread_cond_init">pthread_cond_init()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_cond_destroy">pthread_cond_destroy()</a></tt>.
<br />
<br />
A condition variable can optionally be initialized with attributes in the form of a <tt>pthread_condattr_t</tt>, initialized and freed using <tt><a href="http://linux.die.net/man/3/pthread_condattr_destroy">pthread_condattr_destroy()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_condattr_init">pthread_condattr_init()</a></tt>. You can initialize a statically allocated condition variable using <tt><a href="http://linux.die.net/man/3/pthread_condattr_init">PTHREAD_COND_INITIALIZER</a></tt>.
<br />
<br />
Whether the condition variable is able to be used by multiple processes can be accessed/modified using <tt><a href="http://linux.die.net/man/3/pthread_condattr_getpshared">pthread_condattr_getpshared()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_condattr_setpshared">pthread_condattr_setpshared()</a></tt>.
<br />
<br />
The clock used (identified by a <tt>clockid_t</tt>) for timed waits on a condition variable can be accessed/modified using <tt><a href="http://linux.die.net/man/3/pthread_condattr_getclock">pthread_condattr_getclock()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_condattr_setclock">pthread_condattr_setclock()</a></tt> (though it cannot be set to a CPU clock).
<br />
<br />
Threads can wait for changes in conditions using <tt><a href="http://linux.die.net/man/3/pthread_cond_timedwait">pthread_cond_timedwait()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_cond_wait">pthread_cond_wait()</a></tt>.
<br />
<br />
Threads can signal changes in conditions using <tt><a href="http://linux.die.net/man/3/pthread_cond_broadcast">pthread_cond_broadcast()</a></tt> and <tt><a href="http://linux.die.net/man/3/pthread_cond_signal">pthread_cond_signal()</a></tt>.
JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-63899886351701770332012-10-09T19:59:00.000+01:002012-10-09T20:03:45.299+01:003 Films 1 RoomRecently I was trying to think of all the films I could that are set (almost) entirely in a single room, and a thought struck me; the good ones are some of the best films I know. I'm not sure why this is. Maybe they're a case of constraints breeding creativity? Maybe I just love dialogue-driven films? Maybe I just forget the bad ones? (Although <i>Exam</i> would be a counter-example to that - I want my money back, and I didn't even pay anything to see it*). Here are my three favourite almost-but-not-quite-cult examples.
<br />
<br />
There are no spoilers.
<br />
<br />
<h3>Tape</h3>
<br />
Set in a motel room, two old high-school friends discuss old times, eventually leading to old accusations resurfacing and some very heated moments in such a small space. Ethan Hawke, Robert Sean Leonard and Uma Thurman all give excellent performances.
<br />
<br />
<h3>The Man from Earth</h3>
<br />
Professor John Oldman gets accosted by friends before he can leave his whole life behind, seemingly for no reason and not giving anyone any clues as to where he's headed to next. Pushed for the reasons for such a hasty escape, he confesses to being 14,000 years old, periodically moving when people notice he doesn't age. His academic colleagues try to pick holes in his story, but he makes them doubt themselves with an answer for every question.
<br />
<br />
This film is famous in cult circles for being filmed on a budget of $200,000 (not much for a film these days) and being publicised mostly through word-of-mouth on the internet (or should that be "word-of-keys"?) - the producer even commenting on how positive file sharing has been in getting word of the film out.
<br />
<br />
<h3>Conspiracy</h3>
<br />
A gripping portrayal of the Wannsee Conference, depicting the German high command discussing the "Final Solution" behind closed doors. I'm not much of a history buff, but this film really did engage me. Although this film isn't strictly <i>all</i> in one room all the action (by which I mean dialogue; it feels like action) and a large part of the film does, and the rest is not far away.
<br />
<br />
<i><h3>"Was it a play?"</h3></i>
<br />
For some reason, when I mention to someone that a film is set mostly or entirely in a single room, they often ask if it's an incarnation of a play (as if a screenwriter could ever bring such a restriction upon themselves). So far as I know, of the offerings above only <i>Tape</i> started out in life as a play, although <i>The Man from Earth</i> was subsequently made into one.
<br />
<br />
<br />
<i>* It was on TV - lose one hundred million points if you briefly hated me in your head for downloading films. Win one hundred million points back if you still hate me for preempting you; I like that.</i>JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-61833146226657787222012-10-07T20:38:00.000+01:002012-10-07T20:39:24.170+01:00The Cumbrian Run, or: How Not to Finish Training for a Distance RunWell, the Cumbrian Run is over for another year, yet again leaving me with what feels like the legs of an 80-year-old man. The previous two years have been a bit of a let-down, with me not training as much as I could (especially last year) and being disappointed with my result. Last year's was 2:25, the previous was 2:15. So last year, I vowed to train all year and really boost my time. Which I did. Well... ish.
<br />
<br />
My training started off well, but throughout the year I'd had a couple of breaks of a few weeks or so due to commitments and general laziness. And the last couple of weeks I've been preparing for an exam, as well as just experiencing general apathy for the whole running thing (these things come and go, I guess). This may not be the best time, but priorities are priorities.
<br />
<br />
Anyway, about a month or so ago I would have estimated I could get a 1:50 time fairly confidently, so that's what I was aiming for and the pace I set off at. You couldn't ask for much better weather - still, sunny and fairly cool. Setting off, the first four miles felt pretty good aside from a pain at the back of my ankle - I don't normally get pain here, so that wasn't a good sign. Then <i>Disaster Strikes!</i> in the form of my watch stopping. It's a Garmin Forerunner 305. I've never had any problems with it in the past, and I'm sure I didn't knock it/into anyone, so this is puzzled me somewhat.
<br />
<br />
Anyway, at mile four I reset it and got on with things. The next four miles were pretty painful but I managed to keep up the pace, then the next three I dropped it a bit. I was happy with my last mile though - it was faster paced than my average for the race, and the crowd really helped me finish.
<br />
<br />
In the end, I ended up with a 1:52:13 chip time, which I was fairly pleased with - although I was hoping for under 1:50 going in, I wasn't sure how two weeks of sitting on my arse would affect things (and I still don't - would I have knocked two minutes off my time with extra training? Who knows...) my aim from last year was to get under 2 hours, so I can be happy with that. And I knocked over 32 minutes off my time - if I continue at this rate, I'm on for a pretty good time next year...JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-81547233611246022812012-10-04T20:17:00.000+01:002012-10-04T20:17:38.337+01:00Chilli Confusion: Win for EvolutionI love chilli peppers. They go great in various foods as well as just whole in a nice glass of wine. I also think they (as well as tomatoes) have a neat method of seed propagation: they get eaten by animals and let their seeds be "distributed" when their digestive systems are done with them. This apparently works well for them, since they get (i) a nice new home in foreign climes and (ii) a toasty warm bed of fertilizer to start life off with.
<br />
<br />
However, at same time this has always confused me - while I love them cut up and mixed into my food, if I were your average grazing mammal and decided to try biting into a chunk of chilli for the first time, the intense burning sensation in my mouth would soon put an end to such a behaviour. But if they get propagated in the same way as tomatoes, why would they want to ward off their middle-men?
<br />
<br />
Obviously, there's a perfectly reasonable explanation for this, but neither my teacher at school nor the then-infantile dial-up internet of the time could provide said explanation. Revisiting the problem recently gave me the answers I craved. The heat in chillies arises from evolutionary forces that depend on two major factors that make the answer kind of obvious when you know them, and are a veritable marvel of nature (at least, <i>I</i> think so...).
<br />
<br />
First, there's the fact that while we perceive chillies as having a hot, burning sensation, birds don't. In humans and other mammals, the capsaicin in chillies activates the same pain receptors as direct application of heat does*, but birds don't have these and so are basically unaffected by this.
<br />
<br />
Secondly, the seeds in chillies are vulnerable to mammalian digestive tracts - they are harmed when they pass through them, but not when they pass through avian digestive tracts. Thus, a chilli's heat is how it selects its preferred "thing to be eaten by".
<br />
<br />
So that's my curiosity satisfied, and since it was such a nice evolutionary arrangement I thought I'd share.
<br />
<br />
<br />
<i>* Though it doesn't feel <b>quite</b> the same as actual heat applied to the inside of your mouth (at least to me) because you don't get other consequences of heat (e.g. tissue cell damage and the resultant release of various substances with further consequences). But it really is the same pain receptors that get stimulated (<a href="http://en.wikipedia.org/wiki/TRPV1">these ones</a>, if you're so inclined), just in a different way.</i>
JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-34778973267236108752012-10-02T20:37:00.000+01:002012-10-03T01:28:06.064+01:00Invisibility in Fiction<div>I'd like to comment on a subject I feel very strongly about at a quite personal level. I feel this issue is important, despite being overlooked by many serious authors of various backgrounds and heights. Quite frankly, it gets right on my tits.
<br />
<br />
It is this:
<blockquote><i>
An invisible person would be blind.
</i></blockquote>
This is, of course, ignoring any invisibility obtained through magic or similar, since then an author can also get around this problem with hand-waving and magic. I'm talking more about the sort of thing described in H. G. Wells' <i>The Invisible Man</i>*, or the modern film <i>Hollow Man</i>, where people are described as simply having all their matter turn completely transparent somehow. Fringe cases (such as the girl in <i>The Incredibles</i>) could be argued either way, I guess.
<br />
<br />
Anyway, as you may well know, the big problem is that in order to see, your retina has to absorb light (hence why it's black). But if you're invisible in the way described (i.e. everything that constitutes you/your body is completely transparent), light would pass through your retina, the photoreceptive cells that turn light into neural signals never would get stimulated, and you'd be as blind a completely blind person.
<br />
<br />
There are a few ways you could try and get around this. In the <i>The Invisible Man</i>, H. G. Wells attempts to do so by describing the recently-made-invisible protagonist's retinae** as being translucent in <a href="http://www.online-literature.com/wellshg/invisible/20/">chapter 20</a>: <i><q>an attenuated pigment still remained behind the retina of my eyes, fainter than mist</q></i>.
<br />
<br />
Of course, this state of affairs would be no better than being blind. Without an optically correct cornea, lens and associated fluids that make your eyes so juicy and delicious, light would not get focused on your retina - the best you could hope for is a blur. And even if <i>these</i> weren't completely transparent and had their normal optical properties, without an opaque sclera (the white that surrounds the rest of your eye) light would be arriving at your retina from all other directions, meaning the world would be a big white blur. In order to see, you'd need these bare essentials to be unaffected by any invisibility remedy, but that would leave you with two very distinct and conspicuous white spheres bobbing around at everyone else's eye-level (unless you're very short/tall or you want to go around crouching).
<br />
<br />
As an aside, if you could really do this you may actually see better than you do ordinarily. The photoreceptors in your retina are orientated with their photoreceptive pigment at the <i>back</i> of your eye, meaning the cell bodies as well as various retinal processing cells that aggregate/mediate their responses are all in front of the receptors themselves, in the path of the light. I don't know how much it would <i>actually</i> improve your sight, but it wouldn't be by a vast amount - the best values of visual acuity that have been measured are not much better than the theoretical limit, given the distance between adjacent photoreceptors in the retina.
<br />
<br />
In conclusion, next time I'm watching a film with an invisible person in it, I'll just pretend they suddenly developed extraordinarily good echolocation.
<br />
<br />
<br />
<i>* An attempt at reconciling this problem is made in the book - see later.</i>
<br />
<i>** This is a weird one - saying "retinae" instead of "retinas" sounds weird to me, but writing "retinas" instead of "retinae" looks equally weird. And anyway, both are acceptable and I like writing footnotes.</i></div>JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-83739799560408965022012-10-01T00:09:00.000+01:002012-10-01T13:46:11.954+01:00S205 - Advice for StartersAs an addendum to <a href="http://catdevthoughts.blogspot.co.uk/2012/09/s205-molecular-world.html">a previous post</a> with my thoughts on S205 in general, here's some advice I posted to the S205 group on The Facebook when someone asked. By the way, if you're doing an OU course the Facebook groups are great - there's a much friendlier atmosphere than on the course forums.
<br />
<ul>
<li>
Make sure you understand Book 2 and Book 3 Part 2 <i>really</i> well, as they come up throughout the course.
</li>
<li>
Book 5 Part 2 introduces you to reaction mechanisms and other concepts which are also fundamental to organic chemistry in later parts of the course (Books 5(3), 7 and 10), so it (and the exercises) should also be read thoroughly.
</li>
<li>
The TMA questions tell you which book they're about - read the questions before reading the books. You can then pick up the answer as you're going through, rather than forgetting all about the topic and perhaps missing something important.
</li>
<li>
Conversely, don't think you've got a TMA question complete until you've read the entire book that it's on. Sometimes, general rules that look right are altered by niche conditions/whatever, which is what the TMA is really asking about.
</li>
<li>
Book 8 Part 1 and Book 9 (past chapter 4) can be largely avoided, if you like. I read (but made no notes) on Book 8, and skipped 9 as much as possible. This seemed like a popular option as it's very, very dry! Our TMA question on Book 9 was about a specific chapter, so you can just focus on that.
</li>
</ul>JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-4995506701490978042012-10-01T00:04:00.001+01:002012-10-01T22:54:14.786+01:00S205 - The Molecular WorldSo, I'm part-way to a natural sciences degree with the Open University. My first course was <a href="http://www3.open.ac.uk/study/undergraduate/course/s205.htm">S205 The Molecular World</a> (well, really it was <a href="http://www3.open.ac.uk/study/undergraduate/course/dse212.htm">DSE212</a>, but that was a few years ago and not with a view to my current degree). I like reading what other people thought of courses and the nitty-gritty of them from a student's perspective, so I thought I'd present my views on said course here. So that you know where this view point is coming from, my first degree was in maths and the last chemistry I did was at A-level around 10 years ago.
<br />
<br />
If you're interested, I've also written a brief post on some <a href="http://catdevthoughts.blogspot.co.uk/2012/09/s205-advice-for-starters.html">advice</a> if you're starting this course.
<br />
<br />
<h3>Course Books and Content</h3>
<br />
<b>Book 1</b> of the course is short and sweet (just over 20 pages) and basically acts as an introduction to/taster for the course the case studies (which I'll come back to later).
<br />
<br />
<b>Book 2</b>, <i>Introducing The Molecular World</i> (~70 pages) is an overview of some of the "bread and butter" chemical knowledge that you'd need throughout any chemistry course - the basics of atoms, chemical bonding and chemical/electronic structure, the periodic table and patterns in it, functional groups, molecular shape and reactivity. These are all set out at a fairly basic level, since each topic is taken into much greater detail in later books, but you need at least a grounding in them all to be able to really understand later material. How you get on with this depends on your background; if you've got a good chemical background (e.g. a recent A-level) you'd probably be able to skim through most of this effectively, but it would be understandable (with a little effort) for someone with little or no background in chemistry. Like I said, my last chemistry course was 10 years ago, so things were a bit hazy for me but I felt like this was a good, solid and fairly interesting start to the course.
<br />
<br />
<b>Book 3</b>, <i>The Third Dimension</i> comes in two parts - <i>Crystals</i> (~95 pages) and <i>Molecular Shape</i> (~50 pages). I found the part on crystals to be fairly dry, which was a bit off-putting so early in the course. A fair chunk of it was describing the characteristics of different crystal structures, which went in one ear and out the other; it struck me more as the sort of thing I'd have to memorize before the exam. That part starts with a general discussion of close-packed arrays of spheres and how this reflects the internal structure of metals before moving on to the structure of ionic solids, ionic radii in crystals, molecular crystals and defects in crystal structures.
<br />
<br />
I found the second part of book three much more interesting. It covers molecular shape including isomerism, chirality (molecules with mirror images that aren't identical) including molecules with many chiral centres and chiral centres in ring structures.
<br />
<br />
<b>Book 4</b>, <i>Metals And Chemical Change</i> (~190 pages), seems fairly thick, but a lot of it is a fairly gradual introduction to the core principles you actually need to remember - you can miss a good few chapters out of your revision, for example. It covers thermodynamics, why some reactions happen while others don't (and how to predict which ones will happen), the Born-Haber cycle and the extraction of metals from their ores. It finishes by applying these concepts specifically to group I and II metals.
<br />
<br />
<b>Book 5</b>, <i>Chemical Kinetics and Mechanism</i> is broken down into (obviously) <i>Chemical Kinetics</i> (~95 pages), <i>The Mechanism of Substitution</i> (~45 pages) and <i>Elimination: Pathways and Products</i> (~25 pages). Chemical kinetics links in nicely with both the previous book and the later parts of the book - instead of looking at whether a reaction will happen, it looks at how fast it will happen, what factors affect the speed of a reaction and how this information can be predicted from/used to suggest the mechanism of the reaction. This is probably the most maths-heavy part of the course, and some people found the maths here to be a little bit of a challenge - if you've got a good grasp of algebra and logarithms it's a breeze, otherwise make sure you understand the maths help that's provided and you'll be fine.
<br />
<br />
The second part of book 5 discusses substitution - one of the three major classes of organic reaction mechanism (the others being elimination and addition). It starts off with a general discussion of reaction mechanisms and how to draw/describe them using chemical notation before applying these principles directly to the two types of substitution mechanism, as well as what factors affect which mechanism will be preferred by a reaction.
<br />
<br />
The third part of book 5 is similar to the second part, only discussing a different type of mechanism and without the need for the general introduction to reaction mechanisms. This was nice; it was good to see material that was brief because you'd already learned the underlying principles.
<br />
<br />
<b>Book 6</b>, <i>Molecular Modelling and Bonding</i> (~100 pages) goes into details about the techniques used to predict what shapes molecules will adopt, moving on to quantum chemistry, symmetry, atomic and molecular orbitals (i.e. the shape of electron clouds) and how these affect molecular reactivity, and the bonding in metals and semiconductors. I found the section on how semiconductors work and semiconductor doping particularly interesting.
<br />
<br />
<b>Book 7</b>, <i>Alkenes and Aromatics</i> is broken down into <i>Addition - Pathways and Products</i> (~20 pages), <i>Aromatic Compounds</i> (~35 pages) and <i>A First Look at Synthesis</i> (~40 pages). The first part draws together the principles of reaction mechanisms from book 5 as well as the material on molecular orbitals from book 6 to look at the mechanism of the last of the major classes of organic reaction. Hence the brevity again - you should be familiar with the basic principles from the last two books, this book just squishes together in a fairly neat way to form a coherent whole.
<br />
<br />
The second part of book 7 discusses benzene, the basis of all aromatic compounds, including its orbital structure. It also looks at substitution reactions applied to benzene and how substituents affect these reactions. The final part is a precursor to book 10 - it looks at how synthetic routes (i.e. ways of synthesizing drugs and other chemicals) are planned, the issues involved, how to construct simple synthetic routes and how changes in existing ones will affect the outcome.
<br />
<br />
<b>Book 8</b>, <i>Separation, Purification and Identification</i> is broken down into <i>Chemistry: A Practical Subject</i> (~65 pages) and <i>Spectroscopy</i>. The first part covers methods for the separation and purification steps of this book's title. I found this section fairly dry and we were told before-hand that this section wouldn't be examined, so I didn't study it very in-depth.
<br />
<br />
The second part of book 8 is entirely taught via a computer program - revision notes were available on the course website though. It goes fairly in-depth into the theory behind both infrared and NMR spectroscopy before covering how to interpret both types of spectra for a wide range of organic compounds. The program was interactive, allowing you to try your hand at interpreting IR/NMR spectra, eventually using both together to determine the structure of some quite complex molecules. I enjoyed this part of the course as it was mainly about problem solving - applying a little bit of knowledge in sometimes fairly complicated ways.
<br />
<br />
<b>Book 9</b>, <i>Elements of the p Block</i> is a fairly beefy size (~205 pages) and covers acids and bases as well as hydrogen chemistry, but the bulk of the book is dedicated to patterns and trends in the p block (groups III-VIII, the largest block of the periodic table). I can't really comment too much on this book except that this was by far the least enjoyable part of the course for me - it reads a lot like a huge list of facts to remember, and felt like wading through xenon-infused treacle. In short: Not my cup of tea. I quickly decided I wanted little to do with this material, but this wasn't too much of a problem as the structure of exam meant that certain parts of the course can be skimmed over, if not totally skipped, (so long as you're prepared to do more work on certain other parts of the course) and book 9 was one of them. The upshot was that I read just enough to do the assignment question (though not very well...) on this book before ignoring it.
<br />
<br />
<b>Book 10</b>, <i>Mechanism and Synthesis</i> is the final and largest of the course books, and consists of <i>Carbonyl Compounds</i> (~40 pages), <i>Synthetic Applications of Organometallic Compounds</i> (~35 pages), <i>Radical Reactions in Organic Synthesis</i> (~40 pages), <i>Strategy and Methodology in Organic Synthesis</i> (~75 pages) and finally <i>Synthesis and Biosynthesis: Terpenes and Steroids</i> (~40 pages).
<br />
<br />
The first part discusses the chemistry of the carbonyl group (carbon double-bonded to oxygen), which has a number of uses in reaction synthesis. The second part describes how various organic compounds that include a metallic element can be used in synthesis, another important class of reactions. The third part discusses radical reactions, which involve a different type of bond separation than normally found, including chain reactions. Part four picks up where the last part of book 7 left off, covering approaches for planning more complex synthetic routes. I found this part very enjoyable as it brings together previous parts of the course to solve the sorts of problems found in synthesis. It soon became apparent that chemical synthesis at this level really is more of an art form than a science. The final part of the course covers the synthesis of some naturally occurring compounds.
<br />
<br />
<h3>Course Software</h3>
<br />
I found the software easy to use and helpful, though there was a fair bit to install and some of it was redundant - e.g. there were two chemical drawing programs to install, one an updated version of the other (though I didn't realize this until I'd installed them). The software for book 8 (spectroscopy) was easy to use, and I found the puzzle-solving in it quite fun.
<br />
<br />
However, I was running the course software under Windows XP - some people using Windows Vista or Windows 7 had issues installing and using the software, but so far as I know they seemed to get resolved in the end. The conference software for remote tutorials didn't work very well under Linux (the audio was very broken) but worked fine when I booted into XP.
<br />
<br />
<h3>Assignments and The Exam</h3>
<br />
The assignments were fairly challenging at times, but certainly doable if you cover the material well. The questions were nice and specific (some students have complained of vague, wishy-washy questions in other courses) and there were no essay/experiment write-up type questions (though I got the impression there were in previous presentations). In addition to 6 tutor-marked assignments there were also 2 interactive computer-marked assignments (i.e. you answer numerical/multiple choice questions on a web page). I found the computer assignments useful for jogging my memory; whereas the tutor-marked assignments were all about the recent topics we'd covered, the computerized ones covered the whole course up until that point, and so forced you to revisit earlier material. Some found this a pain, but I was glad for the chance to revise and consolidate everything so far.
<br />
<br />
I found the exam fairly easy to prepare for - there was an exam guide that laid out what sections would be covered. As I said before, this meant you could selectively ignore certain parts of the course - not an awful lot, and you needed a decent grasp on most of it. A certain amount of "question spotting" was possible, but not very much.
<br />
<br />
The exam itself consisted of four sections. The first constituted 40% of the marks and had 12 questions, though you only had to answer 8, which is a fairly generous amount of choice if you ask me. These questions could be taken from any part of the course, but weren't too in-depth. The remaining three sections each had three longer questions, but you only had to answer one from each section (i.e. three questions in total). These were each worth 20% of the total marks for the exam and so required much more detail but, as mentioned before, the organization allowed you to focus revision on a few subjects, within certain limitations.
<br />
<br />
I found the assignments and exam manageable - I ended up with a distinction overall, so I was pretty happy with that.
<br />
<br />
<h3>Overall</h3>
<br />
The course started out moderately paced - the workload picked up towards the end (roughly from book 6 onwards) but I never found it overbearing. I liked how the sections linked together and built upon each other in a nice, logical way (especially the organic chemistry).
<br />
<br />
There were case studies at the end of each book from book 3 onwards. They build on the material in the book but with a more specific "real life" application in mind (e.g. the chemistry of liquid crystals or high-temperature semiconductors). They're not required or assessed at any point - they're purely there for interest. While what I did read was interesting, I didn't have time to read much of them.
<br />
<br />
Overall, I found this course really interesting, even if there were a couple of sections that were a bit dull, and I feel I gained a lot from it.JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com3tag:blogger.com,1999:blog-8401498987555582112.post-70976816179804542542012-09-29T13:58:00.000+01:002012-10-02T01:13:04.517+01:00Vague Essays (in SD329)I'm doing a level 3 (3rd year) science course with the Open University (<a href="http://www3.open.ac.uk/study/undergraduate/course/sd329.htm">SD329</a/>, if you're interested). As part of the assignments, this course includes some essays; not a big deal, I've done a course with 2-3 thousand word essays in it before. But this is the first one where the essay questions were rather vague.<br />
<br />
There was a lot of talk on the course website/The Facebooks about this. Humanities students might be used to these sorts of essay questions, but it seemed to really phase some students when they did their assignments. I was a little put off at first, but then when talking to my tutor about what they were looking for, it became quite clear that, at least in OU science courses:
<blockquote>
<i>If you're given a vague essay question, it's for a good reason.</i>
</blockquote>
This is generally because either:
<ul>
<li>You've only covered so much material on that topic, and writing an essay about it all in the word count would be easy (even if you include some extra research) or, more likely,</li>
<li>They're deliberately giving you scope to waffle on about whatever you want, so long as you keep to the general topic.</li>
</ul>
Once I realised this, it became more easy to think of them as "open-ended" essays as opposed to vague ones; the people who set the assignment don't have some secret, hidden question they "really" want answered, they just want you to talk about anything on the topic.<br />
<br />
This whole thing may seem obvious to you; but it's the sort of implied knowledge nobody has told me (and at least a few others on my course) about. Science students generally like to know <i>exactly</i> what's required when doing an assignment, so being given free reign over a topic can be a little uncomfortable/unnatural at first - but realizing that it's expected makes writing these sorts of essays much easier.JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-57098358321478318112012-09-29T01:05:00.001+01:002012-09-29T13:59:02.662+01:00Running a program and interacting with it.Here's an example of a function, <tt>ccl_popen3()</tt>, that opens a program with the standard streams that you want to write to/read from open. It's from a while ago, I'll brush it up later (in theory).<br />
<br />
Either way, <tt>ccl_popen3()</tt> is the money function, and <tt>main()</tt> demonstrates usage. Compiles on my machine with <tt>gcc popen3.c -o popen3 -std=gnu99 -Wall -Wextra -Werror</tt>.
<pre class="brush:c">
#include <stdlib.h>
#include <unistd.h>
#define READ_END 0
#define WRITE_END 1
#define NENDS 2
#define OPEN_STDIN (1 << STDIN_FILENO)
#define OPEN_STDOUT (1 << STDOUT_FILENO)
#define OPEN_STDERR (1 << STDERR_FILENO)
#define OPEN_ALL (OPEN_STDIN | OPEN_STDOUT | OPEN_STDERR)
#define MAX_STREAMS 3
__attribute__((noreturn))
static void popen3_child(const char *exe,
char *const argv[],
int shell,
int streams,
int pipes[MAX_STREAMS][NENDS])
{
for (int i = 0 ; i < MAX_STREAMS ; i++)
if (streams & (1 << i)) {
close(pipes[i][i == STDIN_FILENO ?
WRITE_END : READ_END]);
if (dup2(pipes[i][i == STDIN_FILENO
? READ_END : WRITE_END],
i) == -1)
exit(127);
}
if (shell)
execvp(exe, argv);
else
execv(exe, argv);
/* Any return from execv()/execvp() is an error. */
exit(127);
}
static void popen3_parent(int streams,
int fds[],
int pipes[][NENDS])
{
for (int i = 0 ; i < MAX_STREAMS ; i++)
if (streams & (1 << i)) {
close(pipes[i][i == STDIN_FILENO ?
READ_END : WRITE_END]);
fds[i] = pipes[i][i == STDIN_FILENO ?
WRITE_END : READ_END];
}
}
/*
* TODO: (i) different functions for child and parent, (ii)
* close original pipe fd's.
*/
pid_t ccl_popen3(const char *exe,
char *const argv[],
int shell,
int streams,
int fds[])
{
int pipes[MAX_STREAMS][NENDS] = {{-1, -1},
{-1, -1},
{-1, -1}};
for (int i = 0 ; i < MAX_STREAMS ; i++)
if ((streams & (1 << i)) && (pipe(pipes[i]) == -1))
goto err;
pid_t pid = fork();
switch (pid) {
case -1: /* error */
goto err;
case 0: /* child */
popen3_child(exe, argv, shell, streams, pipes);
break;
default: /* parent */
popen3_parent(streams, fds, pipes);
return pid;
}
err:
for (int p = 0 ; p < MAX_STREAMS ; p++)
for (int e = 0 ; e < NENDS ; e++)
if (pipes[p][e] != -1)
close(pipes[p][e]);
return -1;
}
#include <poll.h>
#include <stdio.h>
void ccl_poll_input(int fds[3])
{
struct pollfd pollfds[2] = {
{ .fd = fds[STDOUT_FILENO], .events = POLLIN },
{ .fd = fds[STDERR_FILENO], .events = POLLIN }
};
int running = 1;
while (running) {
int res = poll(pollfds, 2, -1);
if (res == -1) {
perror("poll()");
exit(EXIT_FAILURE);
}
struct pollfd *pfd = pollfds;
for (int i = 0 ; i < 2 ; i++, pfd++) {
if (pfd->revents & (POLLERR | POLLNVAL)) {
fprintf(stderr, "poll() error\n");
exit(EXIT_FAILURE);
}
if (pfd->revents & (POLLIN | POLLHUP)) {
char buf[1024];
ssize_t nread = read(pfd->fd,
buf, sizeof(buf) - 1);
if (nread == -1) {
perror("read()");
exit(EXIT_FAILURE);
}
if (nread == 0) {
/* EOF */
running = 0;
break;
}
buf[nread] = '\0';
printf(" Read from %d: \"%s\"\n",
pfd->fd, buf);
}
}
}
}
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <poll.h>
int main(void)
{
int grep_fds[3];
const char *exe = "/bin/grep";
char *const argv[] =
{"grep", "-h", "XX", "jkfdsl", "-", NULL};
pid_t grep_pid =
ccl_popen3(exe, argv, 0, OPEN_ALL, grep_fds);
if (grep_pid == -1) {
perror("ccl_popen3()");
exit(EXIT_FAILURE);
}
/* Write some input to grep's standard input, and close
* it. */
const char *input = "abcd\ngg9XXp\nDqdXXpird\n";
if (write(grep_fds[STDIN_FILENO], input, strlen(input))
!= (ssize_t)strlen(input)) {
perror("write()");
exit(EXIT_FAILURE);
}
close(grep_fds[STDIN_FILENO]);
printf("out: %d, err: %d\n", grep_fds[STDOUT_FILENO],
grep_fds[STDERR_FILENO]);
struct pollfd pollfds[2] = {
{ .fd = grep_fds[STDOUT_FILENO], .events = POLLIN },
{ .fd = grep_fds[STDERR_FILENO], .events = POLLIN }
};
int running = 1;
while (running) {
int res = poll(pollfds, 2, -1);
if (res == -1) {
perror("poll()");
exit(EXIT_FAILURE);
}
struct pollfd *pfd = pollfds;
for (int i = 0 ; i < 2 ; i++, pfd++) {
if (pfd->revents & (POLLERR | POLLNVAL)) {
fprintf(stderr, "poll() error\n");
exit(EXIT_FAILURE);
}
if (pfd->revents & (POLLIN | POLLHUP)) {
char buf[1024];
ssize_t nread = read(pfd->fd,
buf, sizeof(buf) - 1);
if (nread == -1) {
perror("read()");
exit(EXIT_FAILURE);
}
if (nread == 0) {
/* EOF */
running = 0;
break;
}
buf[nread] = '\0';
printf(" Read from %d: \"%s\"\n",
pfd->fd, buf);
}
}
}
int grep_status;
if (waitpid(grep_pid, &grep_status, 0) == -1) {
perror("waitpid()");
exit(EXIT_FAILURE);
}
if (WIFEXITED(grep_status))
printf("grep exited with status %d\n",
WEXITSTATUS(grep_status));
else
// TODO: This better.
printf("grep terminated abnormally\n");
return EXIT_SUCCESS;
}
</pre>JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-49334327781850334262012-09-26T22:08:00.000+01:002013-04-05T11:39:09.079+01:00C on Linux Kickstart<h3>
Overview</h3>
<br />
So, there are plenty of resources for learning how to program in C/C++ (see the end of this post) but here's a quick guide on how to get off to a good start with the practical matters. It's the sort of stuff I wish someone had pointed out to me when I started, since it would have made my life a lot easier.<br />
<br />
While graphical IDE's may make your life easier for large projects, a knowledge of how the core tools used in making programs fit together is essential if you're going to go anywhere with your programming; where more complex tools let you down, you can pick yourself up if you know what they're doing behind the scenes. This is why this guide will start you off using the compiler on the command line.<br />
<br />
<h3>
The command line</h3>
<br />
If you're not already used to using the command line, have a look at <a href="http://tldp.org/LDP/intro-linux/html/chap_02.html">this chapter of Introduction to Linux: A Hands On Guide</a>. You'll need to get comfortable with moving around the file system, creating and listing the contents of directories, etc.<br />
<br />
<h3>
What to use & install</h3>
<br />
The two absolute essentials you'll need are (i) a text editor and (ii) a compiler. You might already be a fan of the text editors <tt>vi</tt> or <tt>emacs</tt>, both popular with programmers, but they can have a steep learning curve. If you've got a graphical Linux distribution it will have a default text editor that will doubtless have syntax highlighting (this makes certain parts of your source code stand out). The defaults for Gnome, Kde and Xfce are <tt>gedit</tt>, <tt>kate</tt> and <tt>mousepad</tt> respectively. So from the command-line, you can open e.g. the file <tt>program.c</tt> from a Gnome desktop by typing the following at a terminal:<br />
<pre> gedit program.c
</pre>
As for a complier, by far the most popular option is GCC. On Debian/Ubuntu, you can install this, as well as all the development libraries you'll need, by installing the <tt>build-essential</tt> package (as root):<br />
<pre> apt-get install build-essential
</pre>
On Fedora/Red Hat, the same effect can be achieved by:<br />
<pre> yum groupinstall "Development Tools" "Development Libraries"
</pre>
If you have another distribution and you're not sure what to install, ask around; if you can compile the <tt>main.c</tt> program in the next section, you have what you need.<br />
<br />
<h3>
How to compile programs</h3>
<br />
To give us something to work with, open your text editor of choice, write the following into it (copy & paste if you like/you get errors) and save it as <tt>main.c</tt>:<br />
<pre class="brush:c">#include <stdio.h>
int main()
{
printf("hello world\n");
}
</pre>
We won't go into what the statements mean (I'll leave that to the tutorials at the end of this post) but this gives us something practical to work with.<br />
Now, you can't run this file directly, you first have to <i>compile</i> it into a file that you can. The simplest way to do this is:<br />
<pre> gcc main.c
</pre>
This will produce a file called <tt>a.out</tt>. You can run this in the usual way (from where you are just after compiling, type <tt>./a.out</tt>). If you like, you can specify the output file to be something different by using <tt>-o filename</tt> (e.g. <tt>gcc main.c -o main</tt>).<br />
<br />
However, you will make your programming life easier if you never compile your programs like this; it will lead you down a bad path. When you compile programs like this, you will let GCC ignore many potentially dangerous practices. Luckily, GCC has a way to alert you to these in the form of its warning options - they control what situations GCC will warn you about, and what situations it will silently ignore. The defaults are pretty lax, and while they may be useful in compiling legacy code you know to work and just want to compile, when learning and/or writing new code, you want your compiler to be as strict as possible. There are an endless variety of options you can fine-tune this strictness to your heart's content if you like, but for now there are three that will serve you very well: <tt>-Wall</tt>, <tt>-Wextra</tt> and <tt>-Werror</tt>. The first two of these enable a group of common warnings, and the last turns all warnings (that would usually just cause the compiler to emit a message about the offending construct) into errors that cause the compiler to flat-out refuse to finish compiling your program until you've sorted the error out.<br />
<br />
These warnings are there for a reason; use them. They will prevent you from making mistakes that will lead to bugs that will just needlessly waste your time. A quite common forum post from new programmers reads essentially "here's a program that doesn't work the way I expect - help!". At least 50% of the time, I can copy + paste the program and compile with <tt>-Wall -Wextra</tt> and the result will tell me what they've done wrong, without even having to look at their program. Why spend time searching for errors when you can let your compiler do it for you?<br />
<br />
In short, always compile your programs with at least <tt>-Wall -Werror</tt> - try it now with our <tt>main.c</tt> above:<br />
<pre> gcc main.c -o main -Wall -Werror
</pre>
You'll notice I've not included <tt>-Wextra</tt> in the above command-line. <tt>-Wextra</tt> can be a bit harsh, especially on new programmers; use it to double-check a program that's not behaving as you expect, or for any code you hope to give to others (including posting it on a forum) for a thorough sanity-check first.<br />
<br />
<h3>
Dealing with error messages</h3>
<br />
There is one golden rule when dealing with error messages. At first you (like me) will probably ignore it, only to learn through bitter experience that it applies no matter how experienced you are. Luckily, it's very simple:<br />
<ul>
<li>Always deal with the <i>very first</i> error message emitted by the compiler.</li>
</ul>
The reason for this is quite simple: once the compiler sees something wrong, it's very likely that later things will make even <i>less</i> sense to it than they ordinarily would. So find the first error the compiler spots, fix it, then try recompiling. Often you will fix several error messages by fixing one actual error.<br />
<br />
The error messages emitted by GCC are always in the format <tt><b>file</b>:<b>line</b>[:<b>position</b>]: ERROR MESSAGE</tt>. If you've tried compiling the above <tt>main.c</tt> with <tt>-Wall</tt> you may have already seen the following:
<pre> main.c:6:1: warning: control reaches end of non-void function
</pre>
This lets us know that the error occurred in <tt>main.c</tt> on line 6 (and was estimated to be at character 1, but that's rarely useful). Exactly what this means you'll learn in time, but in this case we can fix this by turning <tt>main.c</tt> into:<br />
<pre class="brush:c">#include <stdio.h>
int main()
{
printf("hello world\n");
return 0;
}
</pre>
Try recompiling and you'll see no error message.<br />
<br />
When you see an error message you don't understand, don't ignore it or start turning warning options off until it goes away. A quick copy-and-paste it into Google (leaving out any specifics like file names, function names and line numbers) will usually tell you exactly what's gone wrong:
<pre> main.c:6:1: warning: <a href="https://www.google.com/search?q=control+reaches+end+of+non-void+function">control reaches end of non-void function</a>
</pre>
<h3>
Where to go from here</h3>
<br />
There are several good introductions to programming in C on the web, my favourite being <a href="http://cprog.tomsweb.net/cintro.html">this one</a>.<br />
<br />
When you get stuck, forums can be invaluable - there is a Linux-specific forum with an active programming section <a href="http://www.linuxquestions.org/questions/programming-9/">here</a>, and active forums for C and C++ programming <a href="http://cboard.cprogramming.com/c-programming/">here</a> and <a href="http://cboard.cprogramming.com/cplusplus-programming/">here</a>, respectively. You'll get good, fast responses if you include (i) code that demonstrates the problem you're having, (ii) what you think the code should do and (iii) what's going wrong.
JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-59852041618221714802012-09-25T22:28:00.000+01:002012-10-01T00:53:51.083+01:00Seeding Randomness<i>Note: This post is also a demonstration of a simple use of Pthread memory barriers.</i>
<br />
<br />
Recently I read a discussion on choosing a suitable seed for <tt>srand()</tt> - ideally you want something random to seed <tt>srand()</tt> with, and one of the obvious suggestions was to use the current time. This lead me to wonder just how "good" a source of randomness the time is, and how likely two processes/threads are to find the time to be the same in practice.
<br />
<br />
The most coarse source of time is <tt>time()</tt>, which has a resolution of one second. Obviously, any program executed more often than every second is going to see the time to be the same as some other program.
<br />
<br />
Two much better functions in this respect are <tt>gettimeofday()</tt> and the newer <tt>clock_gettime()</tt>; they have a best resolution of one millisecond and one nanosecond, respectively (whether they actually have this resolution depends on your machine). But how likely are they to return the same time?
<br />
<br />
This being the sort of thing that keeps me up late into the night, I set up a program that attempts to spot two calls to one of these functions returning an identical time. My first attempt at just calling either one repeatedly and seeing whether the times were the same showed <tt>gettimeofday()</tt> frequently came out the same if called consecutively, whereas I couldn't get <tt>clock_gettime()</tt> to produce the same result twice. (Obviously since this is a <i>practical</i> (i.e. fudged) test, this result depends on my hardware, OS and possibly how my kernel is configured.) Anyway, undeterred I came up with a scheme using threads.
<br />
<br />
See the listing at the end of the program (with <tt>SYNC</tt> undefined) to see the attempt at just using raw threads. On my machine (stock Ubuntu 12.04 on a Core i5) with NUM_THREADS set to 300 (which was about as high as I could get it before getting memory allocation errors in <tt>pthread_create()</tt>) this program had to run 449 times (on average) to generate one run where only a single match for the time was found.
<br />
<br />
But we can do better than this. Creating a thread takes time - time during which the OS's unpredictable scheduling reigns supreme and with an iron fist, leading our threads to unpredictably diverge. We can force them all to synchronize at a point much closer to the call to get the time by using a Pthread memory barrier. We initialize the barrier with a count of <tt>NUM_THREADS</tt>, meaning any thread waiting at the barrier will wait until <tt>NUM_THREADS</tt> threads (i.e. all of our threads) are also waiting at the barrier before it is allowed to continue execution.* From here, there's a lot less logical "distance" to the <tt>clock_gettime()</tt> call, so we should be able to generate the same time more easily. The barriers in the listing at the end are enabled by passing the <tt>-DSYNC</tt> flag to <tt>gcc</tt> when compiling. Note that we get the time <i>as soon as possible</i> after getting past the barrier, and <i>then</i> check whether our barrier wait was successful, to keep the distance between getting past the barrier and getting the time as low as possible.
<br />
<br />
Using memory barriers, the number of runs we needed to make to get one match was reduced from 449 to 89. Using optimization (<tt>-O3</tt>) reduced this a little further to 62 times. So, my best attempt results in the same time being returned once every (at best) 62 * 300 = 18600 threads.
<br />
<br />
So, the conclusion? If I'm ever after high-quality random numbers for security purposes, I'll turn to <tt>/dev/random</tt> and/or <tt>/dev/urandom</tt>, but for any other application <tt>clock_gettime()</tt> is much simpler and should be more than enough for me.
<br />
<br />
<pre class="brush:cpp">
/*
* same.c
*
* See if we can get clock_gettime() to generate two times
* that are the the same. Can be compiled with -DSYNC to use
* a memory barrier in the thread to try and force the same
* time to be retrieved.
*
* Compile with (where # is the number of threads you want):
*
* $ gcc same.c -DNUM_THREADS=# [-DSYNC] -pthread -lrt
*
* Copyright (C) 2012 John Graham. This source code is
* released into the public domain; you may use it as you
* wish.
*/
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
/* Array into which threads place their times. */
struct timespec times[NUM_THREADS];
#ifdef SYNC
/* Memory barrier at which threads sync (if applicable)
* immediately before getting the time. */
pthread_barrier_t barrier;
#endif
void *thread(void *arg)
{
/* Our argument is the index into 'times'. */
intptr_t index = *(intptr_t *) arg;
#ifdef SYNC
/* Wait at the barrier. This will block until all
* threads are at the same point in their execution. */
int pret = pthread_barrier_wait(&barrier);
#endif
/* Get the time ASAP, *then* check the return value from
* pthread_barrier_wait(). */
if (clock_gettime(CLOCK_MONOTONIC, &times[index]) != 0) {
perror("getting time");
exit(EXIT_FAILURE);
}
#ifdef SYNC
if (pret != 0 && pret != PTHREAD_BARRIER_SERIAL_THREAD) {
fprintf(stderr,
"Waiting at barrier: %s\n", strerror(pret));
exit(EXIT_FAILURE);
}
#endif
return NULL;
}
/* See if *any* of the entries in 'times' are identical. */
int compare_times(void)
{
int nsame = 0;
int i, j;
for (i = 0; i < NUM_THREADS; i++)
for (j = i + 1; j < NUM_THREADS; j++)
if (times[i].tv_sec == times[j].tv_sec &&
times[i].tv_nsec == times[j].tv_nsec)
nsame++;
return nsame;
}
int main()
{
int i; /* Generic index. */
intptr_t nums[NUM_THREADS]; /* Indices to pass to
* threads. */
pthread_t threads[NUM_THREADS];
#ifdef SYNC
if (pthread_barrier_init(&barrier,
NULL, NUM_THREADS) != 0) {
perror("pthread_barrier_init()");
exit(EXIT_FAILURE);
}
#endif
/* Start all our threads, initializing indices as we go
* along. */
for (i = 0; i < NUM_THREADS; i++) {
nums[i] = i;
if (pthread_create(&threads[i],
NULL, thread, &nums[i]) != 0) {
perror("pthread_create()");
exit(EXIT_FAILURE);
}
}
/* Wait until all threads have finished. */
for (i = 0; i < NUM_THREADS; i++) {
if (pthread_join(threads[i], NULL) != 0) {
perror("pthread_join()");
exit(EXIT_FAILURE);
}
}
int times = compare_times();
printf("%d matches\n", times);
return times;
}
</pre>
<br />
<i>* If you want a demonstration that the memory barrier is working, initialize it with a count of <tt>NUM_THREADS+1</tt>. Since this many threads will never wait on the barrier, every thread will sit and wait there indefinitely, doomed to spend the rest of their sad, sorry lives waiting for a thread that doesn't exist (it won't even wake up for <tt>EINTR</tt>, unless this kills the program).</i>JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-86233206916812127052012-09-24T22:43:00.000+01:002012-09-29T14:18:52.494+01:00This BlogBy the way, this blog could be viewed as a continuation/expansion of <a href="http://prognix.blogspot.co.uk/">this earlier one I started</a> which quickly went stale, I think mainly due to its scope being too narrow. Or maybe I just have a limited attention span; I guess time will tell.
<br />
<br />
It's also a place I'll store things I find myself repeating and what a handy reference to, as well as random things (especially code samples) that may be useful to others and that I want to keep. A bit like a public <a href="https://www.dropbox.com/">Dropbox</a> (which I highly recommend, by the way).
<br />
<br />
Some of you might find the previous blog far too geeky/technical - there will be some posts like that in this blog, but there will also be some general/random ones.JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-20021439543077569542012-09-24T22:31:00.001+01:002012-09-24T22:31:15.433+01:00The Sea of Abandoned BlogsThere are many abandoned blogs out there. The most poignant I've seen recently may be <a href="http://www.blogger.com/devrandom.blogspot.co.uk">this one</a>, whose only post ten years ago states that Blogger has given this blogger something to live for.
<br />
<br />
I really hope he was just kidding...JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0tag:blogger.com,1999:blog-8401498987555582112.post-78998150103016122102012-09-24T22:23:00.001+01:002012-09-24T22:47:45.223+01:00The Cost of a DegreeI'm doing a degree with the Open University at the moment, and I get really annoyed when my friends enquire about how they might go about a similar endeavour. Not annoyed at my friends because I have anything against them doing the same - the more the merrier, as far as I'm concerned. I get annoyed that I have to tell them that they may have to pay thousands of pounds more than me for the privilege, and all because I snuck onto the system earlier than they did.
<br />
<br />
For those not familiar with OU degrees, here's basically how it breaks down. You do modules that each carry a certain number of points at a certain level. Undergraduate level 1 corresponds to the first year of a degree, level 2 to the second and level 3 to the third and final year. Usually, you need 120 points from each level for a degree.
<br />
<br />
So, 120 points of part-time, modular study corresponds to one year of a degree, except you don't (necessarily) do 120 points every calendar year (you can, but you'd better have a lot of free time and/or a penchant for late nights). Anyway, the cost of each course is roughly proportional to the number of points it carries, maybe more if it's a residential course or for certain subjects like law and IT. So whether you study four courses worth 30 points or two worth 60 for a year's worth of study, you'll pay roughly the same for the total 120 points for that level.
When I started studying with the OU, the cost for 120 points of study was roughly £1,400.
<br />
<br />
However, since the university tuition fee reform here in the UK, the cost has gone up drastically. Now you can expect to pay £5,000 for 120 points of study - that's a 350% increase in tuition fees. And remember, that's five grand for a single year's worth of study; to do all three years of a degree, you'd now I'd have to pay £15,000 instead of £4,200.
<br />
<br />
Luckily, since I started studying at the lower rate I can continue on these lower fees so long as I keep within certain rules like studying something every year and finishing by a certain time. If I'd started studying one academic year later, or if I step outside the guidelines (e.g. by taking a year off studying) I would/will be on the higher rate, and there's just no way I can afford to do that.
<br />
<br />
"Arrrr!" I hear you cry. "Aye, but there be grants available fer ye if ye wants te study, but ye cannot shell out the doubloons fer it". Well, not for me, since I already have a degree, so I can't get any financial aid for another one. Then again, maybe this is my problem - perhaps I'm being selfish in wanting to broaden my education? Is <i>another</i> degree really just a selfish endeavour I should be made to pay through the nose for? I don't know; I'm just glad I started when I did.JohnGrahamhttp://www.blogger.com/profile/01560079474925666017noreply@blogger.com0