I would recommend creating one lesson plan for your whole book, and then using the "Generate Multiple" option for creating your assignments. When it asks for the type of assignments, you can choose either Chapter Subsection or Lesson Subsection. When you get in there you can see what the Assignment names might look like.
Now, for purposes of generating assignments, it will ask you for the number of subsections (lessons) per chapter. I know that each chapter is different, but try to choose an average number, because OLLY will initially generate that number of lessons per chapter. But don't worry, because a couple of screens later, it will allow you to go down the list and edit the number of lessons per chapter (and also enter topics if you wish).
You can try generating your list different ways, and if you don't like how it looks, just hit "undo" (Command - Z) right away and try again.
I hope this helps!