Much of mathematics is written in a peculiar pidgin. If you wish to write in standard English, here are a few things to avoid.

Don't use "so that" when you mean "such that"

Here are the definitions.

so that

1. In order that, as in I stopped so that you could catch up.
2. With the result or consequence that, as in Mail the package now so that it will arrive on time.
3. so ... that. In such a way or to such an extent that, as in The line was so long that I could scarcely find the end of it.

From dictionary.com

such that

1. adj : of a degree or quality specified (by the "that" clause); their anxiety was such that they could not sleep. (dictionary.com)
2. A condition used in the definition of a mathematical object. For example, the rationals can be defined as the set of all m/n such that n is nonzero and m and n are integers . (mathworld.wolfram.com)

Examples

1. We require x to be a rational number so that mx is an integer for some m. [Correct]
2. We require x to be a rational number so that 3x is an integer. [Incorrect; should be such that 3x is an integer.]
3. Let H be a discrete subgroup of the Lie group G so that G/H is compact. [Incorrect --- not all discrete subgroups of Lie groups have compact quotient; this is from the Annals of Math., 107, p313.]
4. Let N and N' be submodules of a module M such that N contains N', so that N/N' is a submodule of M/N'. [Correct! From Steps in Commutative Algebra.]

Briefly, if omitting the "that" from "so that" renders the sentence nonsense, then it was already nonsense, and you should have used "such that". You won't find "so that" among lists of commonly misused expressions because only mathematicians commonly misuse it. Probably the error arose from the influence of German on American (mathematical) English, since the two are not distinguished in German.
[From Matthias Künzer: They are distinguished:
Sei x in Q, so daß es ein m in Z\{0} mit xm in Z gibt.
Sei y in Q so, daß 3y in Z liegt.
Not very many German writers care. "After all, it's just a comma."]

Write "Gowers's Weblog", not "Gowers' Weblog" or "Gower's Weblog" (Gowers)

[Incidentally, The Princeton Companion to Mathematics is written in unusually good English.]

See:

Use an apostrophe plus -s to show the possessive form of a singular noun, even if that singular noun already ends in -s. (grammar.about.com)

Form the possessive singular of nouns by adding 's. (Strunk and White, The Elements of Style, Elementary Rules of Usage, Rule 1.)

The possessive of most singular nouns is formed by adding an apostrophe and an s. (The Chicago Manual of Style, 7.17)

If you write "Gowers' Weblog", not only will you commit a solecism, but you will mislead a literate reader into thinking that you are referring to the weblog of two or more persons called Gower.

For a truly gruesome error, see BAMS 46.4 (2009), p601, line 8, where one finds: "the Langland's program". (Should be "the Langlands program" or "Langlands's program".)

In brief: the rule is add 's (not ') to singular nouns, except that some authorities allow a few exceptions. In my reading I virtually always see 's used except in mathematics.

Don't write "verify" when you mean "satisfy"

"Verify" is often misused by mathematicians for "satisfy", especially by those whose native language is French. For example, Roos (2006) writes: "The reason for this is that AB4* is rarely verified for them." He means satisfied. It is possible for a condition to be always satisfied but rarely verified (for example, the commutativity of a certain class of diagrams).
verify means to prove the truth of; satisfy means to fulfill the requirements of. For example, I believe the function satisfies the Leibniz condition, but I haven't verified [i.e., checked] this.

Don't use "associate to"

Instead use "associate with" or "attach to", whichever is more appropriate. In English, you may associate with gangsters, or attach yourself to the Crips, but you may not associate to either: "associate to" is not English (native French and Italian speakers please take note). Alas,this particular illiteracy has become almost standard in scientific journals -- where once we had two expressions "attach to" and "associate with" with distinct uses, we now have only one "associate to = attach-to-associate-with". [Even Google Translate gets this right: it translates "associé à" correctly as "associated with".]

Normally pure mathematicians are relatively respectful of grammar, but many of them have adopted the habit of using the dreadful phrase "associated to" when they seem to feel that "associated with" has not a sufficiently specific flavour. I am at a loss to understand why they do not use the perfectly grammatical "assigned to" instead. (Penrose, The Road to Reality, p.354.)

The verbs "permit", "allow", "reduce" normally require objects

For example, the following are incorrect:
The preceding result permits (or allows) to assume that x is positive. [Should be: permits (or allows) us to assume.]
By taking a shortcut, we reduce to 10 miles. [Should be: By taking a shortcut, we reduce the distance to 10 miles.]
We use the local theory to construct the formal group associated to the Neron-model, which then allows to simplify the global steps (Faltings 2008, p93). [Should be: ... formal group attached to... allows us...]

Let now us praise famous men.

Only mathematicians write "Let now" --- real people write "Now let" or "Let us now".

Be careful with your use of "any"

The word "any" can mean "one, some, every, all" (see any dictionary). Sometimes it is clear from the context which of these you mean, but usually it is better to choose a more precise word. Compare (in increasing order of ambiguity):

If it is true for one element, then it is true for all.
If it is true for any element, then it is true for all.
If it is true for one element, then it is true for any.
If it is true for any element, then it is true for any.

The next sentence, from an actual article, is truly ambiguous:

... he introduced the "associate form" of a cycle, apparently the first treatment valid in any characteristic.

Because of its ambiguity, Halmos decreed that "any" should never be used in mathematical writing.

Be careful with quantifiers and negatives

The following statements are equivalent:

1. a(n) \neq 0 for all n.
2. a(n) is not equal to zero for all n.
3. a(n) is not zero for all n.
4. There exists an n for which a(n) is not zero.

So don't use the first to mean "no a(n) is zero"; instead write "no a(n) is zero" (or at least put a comma before the "for all").

Write sentences that can be parsed by a nonexpert

Serre gives the example of an article that announces its main result in the form:

formula A = formula B = formula C

(no words). Only someone very expert in the field will recognize that the main result is "formula A = formula B" --- the second equality is only an aside.

Don't write "issue" or "challenge" when you mean "problem"

The first is an illiterate euphemism invented by software companies to avoid admitting that their stuff doesn't actually work. The second is feel-good English best left to the New Age crowd.

Don't write "address" when you mean "solve", "examine", "study", or ...

Instead, write "solve", "examine", "study", or whatever it is you actually mean. For example, don't write "the conjecture is successfully addressed"; write "the conjecture is proved" (if that is what you mean).

The last two are part of a plague that is spreading into mathematics. Soon people will be writing "Matiyasevich addressed Hilbert's tenth issue", which should mean that Matiyasevich spoke to Hilbert's tenth child...

Be wary of using :=

Personally, I consider the use of "A := B" for "B is defined to be A" or "B equals A by definition" to be ugly programming jargon (especially ugly if the symbols aren't vertically centred--use the package mathtools to fix this), but if you do choose to use it, use it only when the colon genuinely adds something. Note that "We define A:=B" is shorthand for "We define A is defined to be B".