Improving Your Copy
Copy - Material, such as a manuscript, that is to be set in type.
The next feature I want for article mode is for it to be able to help you to write good copy. Sounds impossible? Well, copyblogger has an article with some rules you can follow.
To get started, just delete the word very. While you’re at it, get rid of cowardly qualifiers like "it seems" or "just might be" or "could be considered." Wimpiness is for first drafts. If it helps you to visualize Hans and Franz at this point, feel free.
I’d like something that highlights these cowardly qualifiers and perhaps also sentences that are too long (like this one) along with commands to move between the sections of problematic copy.
(defvar am-max-sentence-len 25) (defvar am-copy-positions nil) (defvar am-current-pos -1) (defvar am-copy-qualifiers (concat "very\\|" "it seems\\|" "just might be\\|" "could be considered\\|" "that\\|" "really\\|" "just"))
am-copy-positions will store the locations as a vector that we can move forwards and backwards along and am-current-pos will index into that vector.
(defun am-add-copy-issue (start end) (let ((marker-start (make-marker)) (marker-end (make-marker))) (set-marker marker-start start) (set-marker marker-end end) (push (cons marker-start marker-end) am-copy-positions)) (am-add-overlay start end 'am-copy-face))
When a problem is found there are two jobs to do - store the location and add the overlay. In the interests of DRY there is a single function, am-add-copy-issue that does both. (am-add-copy-issue ...) stores the start and end of the suspect copy as markers1. I don’t really need the end position at the moment but I am considering of adding a command to easily delete the issue too.
(defun am-highlight-long-sentences () (interactive) (save-excursion (let (start end matches) (goto-char (point-min)) (setq start (point)) (while (< start (point-max)) (forward-sentence) (setq end (point)) (when (> (count-matches *am-re-word* start end) am-max-sentence-len) (am-add-copy-issue start end)) (setq start end))))) (defun am-highlight-qualifiers () (save-excursion (goto-char (point-min)) (while (re-search-forward am-copy-qualifiers nil t) (am-add-copy-issue (match-beginning 0) (match-end 0)))))
am-change-copy-highlight is a helper function for moving between the sections of bad copy. It checks if the bad copy check is active and if we are at the first / last section before amending the index. am-next-copy-highlight and am-previous-copy-highlight then call this with a -1 or a +1 offset respectively.
(defun am-change-copy-highlight (offset) (let (newpos) (when (numberp am-current-pos) (setq newpos (+ am-current-pos offset))) (cond ((not am-copy-positions) (message "Error: copywriting check not active")) ((or (< newpos 0) (> (1+ newpos) (length am-copy-positions))) (message (format (concat "Error: am-current-pos not " "a valid index (%s %s)") newpos offset))) (t (progn (setq am-current-pos newpos) (goto-char (car (elt am-copy-positions newpos)))))))) (defun am-next-copy-highlight () (interactive) (am-change-copy-highlight -1)) (defun am-previous-copy-highlight () (interactive) (am-change-copy-highlight 1)) (defun am-reset-copy-highlight () (interactive) (setq am-current-pos (length am-copy-positions)) (am-change-copy-highlight -1))
am-check-copy-start resets the positions and highlights the long sentences and suspect qualifiers. vconcat can be used for converting lists into vectors. am-check-copy-stop resets the variables and deletes the overlays.
(vconcat &rest SEQUENCES)
Concatenate all the arguments and make the result a vector.
The result is a vector whose elements are the elements of all the arguments.
(defun am-check-copy-start () (interactive) (setq am-copy-positions nil) (am-highlight-long-sentences) (am-highlight-qualifiers) (setq am-copy-positions (vconcat am-copy-positions) am-current-pos (length am-copy-positions)) (am-reset-copy-highlight)) (defun am-check-copy-stop () (interactive) (setq am-copy-positions nil am-current-pos nil) (am-delete-overlays))
And which keys should I use for each of the features? Perhaps these ones?
(define-key article-mode-keymap (kbd "C-x C-l") 'am-check-copy-start) (define-key article-mode-keymap (kbd "C-x C-n") 'am-next-copy-highlight) (define-key article-mode-keymap (kbd "C-x C-p") 'am-previous-copy-highlight) (define-key article-mode-keymap (kbd "C-x C-r") 'am-reset-copy-highlight)))
1. Markers are updated so that they keep pointing to the same thing when the text before them changes. They are necessary in this case as I imagine that the bad copy will often need to be changed.
Count Active Region Words - Another Article Mode Enhancement
As I mentioned previously I wanted to add a feature to article mode that would count the number of words in an active region. I can’t think of an immediate practical purpose, but I am frequently curious how long a paragraph is. If the feature has been implemented correctly, I can tell you that this one, for example, is around sixty-nine words long depending on your definition of a word.
To fit in with the rest of the mode, it would be nice if the information was automatically displayed when the region is active. There are two likely candidate hooks - activate-mark-hook and deactivate-mark-hook which look promising. However, on experimentation with activate-mark-hook, it didn’t really work as I wanted.
activate-mark-hook documentation
Hook run when the mark becomes active. It is also run at the end of a command, if the mark is active and it is possible that the region may have changed.
What I really want is that when a region is active I may need to recalculate the number of words in that region whenever a command is executed.
When the mark is active, record the start and end of the region and restart the timer to update the info window.
post-command-hook is ideal for this and I can literally express my intent line for line in the code. I’m not happy with my naming convention for the functions that are run by the hook variables as they can be mistaken for hooks themselves but I’m not feeling very inspired.
(defun am-post-command-hook () (when mark-active (setq am-region-beginning (region-beginning)) (setq am-region-end (region-end)) (am-restart-timer)))
am-region-beginning and am-region-end are variables we defined earlier and I decided to immediately remove the region word count when the mark becomes inactive.
(defvar am-region-beginning nil) (defvar am-region-end nil) (defun am-deactivate-mark-hook () (setq am-region-beginning nil) (setq am-region-end nil) (when (timerp am-timer) (cancel-timer am-timer)) (am-update-info))
am-update-info is then updated to add the new information if it is available. The statistics need to be gathered before we switch into the info buffer as otherwise we are uselessly measuring the statistics of the info buffer itself. It might have been nicer to write this all into a string before calling a function that just updates the info buffer here. But, hey it works.
(defun am-update-info () ... (let ((words (count-matches *am-re-word* (point-min) (point-max))) (keywords (if (= (length am-keywords-regex) 0) 0 (count-matches am-keywords-regex (point-min) (point-max))))) ... (insert (format "# Words : (%s%s)\n" (if region-words (format "%s / " region-words) "") words)) ...)
The code for restarting the timer is abstracted out into its own function as I seem to be calling it from multiple places. The DRY folks should be pleased.
(defun am-restart-timer () (when (timerp am-timer) (cancel-timer am-timer)) (setq am-timer (run-with-timer am-update-time nil 'am-run-timer-hook)))
All that now remains is to add the hooks in article-mode-start and remove the hooks in article-mode-stop.
(add-hook 'post-command-hook 'am-post-command-hook nil t) (add-hook 'deactivate-mark-hook 'am-deactivate-mark-hook nil t) (remove-hook 'post-command-hook 'am-post-command-hook) (remove-hook 'deactivate-mark-hook 'am-deactivate-mark-hook)
The latest code is in the usual place. Thanks for reading.
Emacs Faces - Using Overlays
I mentioned last time that (add-text-properties ...) doesn’t usually work in a buffer with font-lock activated. How then can we highlight keywords in the general case?
I didn’t know the answer myself before I started implementing article mode, so I looked at a minor mode that does correctly highlight words in a font-locked buffer - flyspell-mode I opened the code by typing C-h f flyspell-mode<RET> and then clicking on the `flyspell.el' part of the resultant document.
flyspell uses overlays to highlight misspelled and duplicate words. It specifies the properties of the highlights using defface. You can see in the code below I’m a lazy commenter.
(defvar am-overlays nil) (defface article-keyword-face '((((class color)) (:background "yellow" :foreground "black" :bold t)) (t (:bold t))) "Article mode keyword face")
Let me explain defface as I found the relevant emacs manual section slightly confusing on first reading. A face can appear differently depending on the properties of the terminal it is displayed on. e.g. a vt100 terminal won’t have colours available whereas a GUI will.
I’ve highlighted a couple of sections here - ((class color)) means the face will look like this on a colour terminal. The (t …) section is a fall through and matches if nothing else did.
(defun am-add-overlay (begin end) (let ((overlay (make-overlay begin end))) (overlay-put overlay 'face 'am-keyword-face) (push overlay am-overlays))) (defun am-delete-overlays () (dolist (overlay am-overlays) (delete-overlay overlay)) (setq am-overlays nil))
Okay, here I’ve defined functions to highlight a single region and to delete all overlays as this will tie in well with the original functions that made similar calls to (add-text-properties ...) and (remove-text-properties ...)
You can see how the new functions substituted in (am-highlight-keywords) and (am-remove-highlight) below.
Before
(while (re-search-forward am-keywords-regex nil t)
(add-text-properties (match-beginning 1)
(match-end 1)
'(face highlight)))
After
(while (re-search-forward am-keywords-regex nil t)
(am-add-overlay (match-beginning 1) (match-end 1)))
Before
(defun am-remove-highlight () (remove-text-properties (point-min) (point-max) '(face highlight))
After
(defun am-remove-highlight () (am-delete-overlays)
Here is the new article-mode.
An Emacs Minor Mode
(aka the Article Mode conversion)
Emacs has a convenience macro for defining minor modes called (appropriately enough) (define-minor-mode ...). It takes a few handy keywords, e.g. :lighter to define that identifier in the mode line and :keymap to define the keymap and the body is executed every time article mode is activated or deactivated.
The new core of my article mode now looks like this:
(define-minor-mode article-mode "Minor mode to assist writing articles." :lighter " Article" :keymap article-mode-keymap (if article-mode (article-mode-start) (article-mode-stop)))
What else has changed?
- Keyword highlights are only removed after a change, rather than after any keystroke (e.g. scrolling)
- The info window is deleted on exiting article mode
And that is about it. I’ve also made a few tweaks to make the updating of the info window a little more reliable.
Let me show you the -start and -stop functions.
(defun article-mode-start () (am-add-info-window) (am-update-info) (add-hook 'after-change-functions 'am-after-change-functions-hook nil t) (add-hook 'am-timer-hook 'am-update-info nil t) nil) (defun article-mode-stop () (remove-hook 'after-change-functions 'am-after-change-functions-hook) (remove-hook 'am-timer-hook 'am-update-info) (delete-window am-info-window) (setq am-info-window nil))
As you can see, they are both pretty simple. The -start function adds the info window and immediately updates it so that it is current. Then it adds a buffer-local1 hook to after-change-functions. This hook kicks off a timer that runs the hooks in am-run-timer-hook when emacs is idle for two seconds2. The reasoning behind this is that (am-update-info) will eventually be a fairly expensive function and we don’t want to run it after every change in the buffer.
(defun am-run-timer-hook () (run-hooks 'am-timer-hook)) (defun am-after-change-functions-hook (begin end length) (when article-mode (when (timerp am-timer) (cancel-timer am-timer)) (setq am-timer (run-with-timer am-update-time nil 'am-run-timer-hook))))
Unfortunately, the conversion to a minor mode has highlighted (if you’ll pardon the pun) a shortcoming of the original implementation. We used (add-text-properties ...) to highlight the keywords but this doesn’t generally work if font-lock is activated in the buffer. We’ll discuss how to fix this in the next post.
Here is the updated article-mode.el.
1. The fourth parameter to (add-hook ...) makes a hook buffer-local if it is non-nil.
2. Why not use idle timer? I only want the hook to run after activity in the article mode buffer.
Plans to Improve Article Mode
Okay, I can’t leave well enough alone. I’ve spent around two hours writing my article mode. This has (depending on your point of view), either wasted two hours, or saved $19.95 (for an alternative tool). Now I’m about to pour countless hours more of my already scarce free time into perfecting the tool. Um, hooray?
First things first. It is probably a good idea to have a plan where I am going with this. So, which additional features do I want? The first problem is that my workflow is inefficient. I spend far too long going from article to HTML.
I use muse and htmlize for my HTML generation. However, htmlize is not currently compatible with Emacs23 (and I love the beautiful fonts). Therefore, I have been writing my articles in article mode and then switching to Emacs22 to generate the HTML. Pretty darn slow. Fortunately it is easy to fix. There is a patched version of htmlize that works with Emacs23. And a sigh of relief was heard all round.
The next thing to do, as has been commented on, is to port article mode to a minor mode so it can co-exist with muse. After that, which additional features would I like?
- One-key HTML generation would be handy.
- It would be nice to be able to count the words just in the active region.
- I should have some form of grammar analysis available.
- I would like to have some default empty templates, maybe with my bio at the bottom of the article.
- The fixed window at the bottom of the article is inflexible. Might it be worth changing this to some form of floating frame à la speedbar?
I also considered incorporating kind of article spinning tool but I don’t think I agree with the idea of generating lots of similar articles so I won’t be facilitating that.
I’ll be blogging about updates tothe article writer over the next few posts. Does anyone have any suggestions for additions to the perfect article writing tool?
Keyword Optimization - Extending Emacs for SEO
I read an interesting post about a tool for writing keyword optimized articles on my new favourite blog. Now, potpiegirl has a lot of great information, and every so often she talks about a handy tool and throws a few affiliate links in there. Nice marketing strategy! Anyway, the tool’s features that she mentioned do sound handy.
- Check keyword density
- Check spelling
- Show word count
- Save content as either text or HTML
- Highlight keywords
I like to use a pretty geeky tool called emacs for almost all my content generation. The tool was created in the seventies before any such thing as keyword optimization. However, it fits me like an old glove - I feel really productive when using it. And it does allow a dedicated user to add almost any functionality they might want.
There are already third party extensions to add on the fly spellcheck and easy HTML generation which just leaves word count keyword density and highlighting.
The basic plan is to have a small frame at the bottom of the editor which constantly updates with the number of words and keyword density. Highlighting will be controlled by a key combination and disabled when any other key is pressed.
We need to explain to emacs what a word is (so it can count them) and we also need a name for the buffer at the bottom of the window.
(defconst *ac-re-word* "\\(\\b[-A-Za-z0-9_]+\\b\\)") (defconst *ac-info-buffer-name* "*article-info*")
We will need a timer as we won’t want to update the information after every keystroke. Providing a hook for the variable means that other people will be able to add their own functionality when the timer fires.
We also need somewhere to store the keywords and a variable to indicate whether the keywords are highlighted or not.
(defvar ac-timer nil) (defvar ac-timer-hooks nil) (defvar ac-info-buffer nil) (defvar ac-keywords nil) (defvar ac-keywords-regex nil) (defvar ac-highlighting-on 0)
We’ll want a way to add and remove keywords that are relevant to the article.
(defun ac-set-keyword-regex () (let ((re (mapconcat (lambda (e) e) ac-keywords "\\|"))) (setq ac-keywords-regex (if (> (length re) 0) (concat "\\(" re "\\)") "")))) (defun ac-add-keyword (keyword) (interactive "sKeyword: ") (if (member keyword ac-keywords) (error (format "Error: keyword %s is already present" keyword)) (push keyword ac-keywords) (ac-set-keyword-regex))) (defun ac-remove-keyword (&optional keyword) (interactive) (when (not ac-keywords) (error "Error: no keywords set")) (unless (stringp keyword) (setq keyword (ido-completing-read "Keyword: " ac-keywords nil t nil))) (setq ac-keywords (remove keyword ac-keywords)) (ac-set-keyword-regex))
Highlighting the keywords takes three functions - one to add the highlight, one to remove it and one to toggle it.
(defun ac-highlight-keywords () (when (not ac-keywords) (error "Error: no keywords set")) (save-excursion (goto-char (point-min)) (while (re-search-forward ac-keywords-regex nil t) (add-text-properties (match-beginning 1) (match-end 1) '(face highlight))) (setq ac-highlighting-on 2))) (defun ac-remove-highlight () (remove-text-properties (point-min) (point-max) '(face highlight)) (setq ac-highlighting-on 0)) (defun ac-set-highlight (&optional value) (interactive) (when (not value) (setq value (if (> ac-highlighting-on 0) nil t))) (if value (ac-highlight-keywords) (ac-remove-highlight)))
Populating the information buffer is fairly simple - we clear any text that was there previously and redraw it all. We only need to report keyword density if there are actually some keywords defined.
(defun ac-update-info () (save-excursion (let ((words (count-matches *ac-re-word* (point-min) (point-max))) (keywords (if (= (length ac-keywords-regex) 0) 0 (count-matches ac-keywords-regex (point-min) (point-max))))) (set-buffer ac-info-buffer) (erase-buffer) (insert (format "# Words : %s\n" words)) (when (and (> words 0) (> (length ac-keywords-regex) 0)) (insert (format "# Keywords : %d\n" keywords)) (insert (format "Density : %.2f%%\n" (* (/ keywords (float words)) 100.0)))) (insert (format "Keywords : %s\n" ac-keywords)))))
ac-add-info-window takes care of adding the appropriately sized frame at the bottom of the window.
(defun ac-add-info-window () (interactive) (delete-other-windows) (split-window-vertically (- (window-height) 6)) (other-window 1) (setq ac-info-buffer *ac-info-buffer-name*) (switch-to-buffer ac-info-buffer) (ac-update-info) (other-window -1))
We set a timer to fire whenever emacs has been idle for half a second. The timer runs all the hooks in the hook variable to allow additional user customisation.
(defun ac-run-timer-hooks () (run-hooks 'ac-timer-hooks)) (defun ac-add-timer () (setq ac-timer (run-with-idle-timer 0.5 t 'ac-run-timer-hooks)))
This is fairly standard boiler plate for any derived mode I write for emacs.
(defvar article-mode-hook nil "Hook run when entering article mode.") (defvar article-mode-map nil "Keymap for article major mode.") (define-derived-mode article-mode text-mode "Article" "Major mode for editing articles Special commands: \\{article-mode-map}")
Set up the special keys which the mode uses - one for highlighting keywords and one each for adding and removing them. CTRL-C + <key> is an emacs convention for user commands (so perhaps I shouldn’t be using it - oops! Force of habit)
(if article-mode-map nil (progn (setq article-mode-map (make-sparse-keymap)) (define-key article-mode-map (kbd "C-c a") 'ac-add-keyword) (define-key article-mode-map (kbd "C-c d") 'ac-remove-keyword) (define-key article-mode-map (kbd "C-c h") 'ac-set-highlight)))
Figuring out how to disable the highlight after any key was pressed took the most time. There is an emacs hook that allows you to insert a function each time a key is pressed. However, the obvious thing to do - disabling the highlight, doesn’t work as adding the highlight is a command. It would add the highlight and then the hook would run which would remove it immediately.
I worked around this by setting the highlight flag to 2 when the highlight is set. Then, each time the post command hook fires, this value is decremented. This is more for micro-efficiency so that when the flag is set to 0 we don’t have to remove the highlight on every keypress.
(defun ac-post-command-hook () (cond ((>= ac-highlighting-on 2) (setq ac-highlighting-on 1)) ((and (= ac-highlighting-on 1) (equal major-mode 'article-mode)) (ac-remove-highlight))))
Enabling article mode sets a bunch of things up such as adding the information frame, enabling the on the fly spellcheck and enabling longlines mode. Longlines mode causes the words to wrap around, just as a normal word processor which emacs normally doesn’t do.
(defun ac-article-mode-entry () (add-hook 'post-command-hook 'ac-post-command-hook t) (ac-add-info-window) (flyspell-mode 1) (longlines-mode 1)) (add-hook 'ac-timer-hooks 'ac-update-info) (add-hook 'article-mode-hook 'ac-add-timer) (add-hook 'article-mode-hook 'ac-article-mode-entry) (provide 'article-mode)
And there you go - all done. The majority of the extension was finished in an hour. Getting the highlight to disappear after any keypress (which was an entirely unnecessary feature, but something I wanted) was a bit fiddly and took another hour. But now it’s complete and emacs becomes the perfect tool for writing keyword optimized articles. Yay!
Download article-mode-major.el.
Note: this has now been superseded by a minor mode version. The updated code can be downloaded here.