CRUCIVERB.COM

User

Welcome, Guest.
Please login or register.
 
 
 
Forgot your password?

Navigate

Resources

Donations


You can help support this site by making a small donation using either a PayPal account:

or with a major credit card such as:

 

 

Click here for details.

Author Topic: Understanding, obtaining, curating word lists (I use Crossfire)  (Read 13221 times)

flight

  • Newbie
  • *
  • Posts: 38
I'm new to crossword construction after solving hundreds of NYT puzzles over the years and I'm having a difficult time understanding how to use word lists/dictionaries, particularly in CrossFire.

I've done quite a bit of digging and haven't found a clear overview of how to obtain, manage, curate, score (or not) word lists, and, for that matter, I don't really understand the broader concept and finer points about scores, scoring words, using scored word lists, etc.). I understand the general concept--and I suppose that a lot of this will clear up as I progress in my craft as a constructor--but I don't understand the Best Management Practices and right now I get lost pretty quickly when it comes to the details and nuance. I'm also nervous that my lack of understanding will cause me to establish bad habits or just not have an intelligent strategy about obtaining/using/curating/managing these lists from the beginning--and then have to go back and undo my mistakes or fix some structurally FUBAR'd situation at a later point (I'm all for "learn by doing", I just want to learn by doing intelligently, not ignorantly). I hope someone here can help me with this, perhaps point me towards a more detailed discussion somewhere, or show me other helpful resources, or just be willing to converse with me about this directly.

I bought Crossfire, then I installed the cruciverb-l ALL.txt word list and also another word list or two from online sources. I successfully moved them into the Crossfire (Preferences-->Dictionary-->Add) and they appear in my list of dictionaries. But now what? I don't understand the concept of ordering these dictionaries in the Preferences list. Why position one above or below another if they're all going to be used together in a single list (are they)? Why keep the MyEdits.dict dictionary above the others (or not)? Do I need to merge dictionaries? What exactly happens when you merge dictionaries? Why would I need to use the Edit-->Save Dictionary command? Which Dictionary does it save, how do I choose this (DO I even get to choose this?)? What are the ramifications of having multiple word lists wrapped into one (does this even happen?) when some are scored using 0-50 scale, others use 0-100, and some don't score at all? Should I merge these together? Won't that Frankenstein words of different scoring scales into each other and kinda render the whole point of using these scores moot, since it will all be unstandardized? While completing fill, if I add words to a grid on the fly that aren't in "my dictionary", do these words automatically go into my dictionary or must I add them/save them? But what does "my dictionary" even mean, when I have multiple dictionary files to choose from? What about if I want to eliminate all the scores from all the entries, must I do this one word at a time? In each different dictionary? Or can I do one big sweep and zero (or 100) them all out in one fell swoop? How do I know what score to give words that aren't scored, if I want to do so?

Thus far, I've inferred a fair amount about these questions, and I've read several resources online as well as the documents that Beekeeper Labs provides ("An Introduction to CrossFire" and "Making a Puzzle with CrossFire"). These were helpful and I gleaned some ideas about all of this word list/dictionary business, but these sources didn't really provide the big picture or the nuance, and reading them hasn't helped me enough to really understand the architecture of word lists/dictionaries and how to use them intelligently and, instead, has only raised more questions, like how do I properly set up and start using my dictionary(-ies?) and word lists from the ground up? How is a Dictionary different from a Word List (is it?)? Is it not good enough to merely add multiple word lists to CrossFire, click the check boxes in Preferences, and then start constructing?

I don't mean to sound daft, I'm just insecure without a clear overview of the whole business. Can someone help me with this?

Thanks in advance!


flight

  • Newbie
  • *
  • Posts: 38
Re: Understanding, obtaining, curating word lists (I use Crossfire)
« Reply #2 on: June 18, 2020, 11:44:33 PM »
Yeah, I read that, too but forgot to mention it above. It isn't really very helpful. Just raised more questions. :-/

stephenbishop

  • Newbie
  • *
  • Posts: 12
Re: Understanding, obtaining, curating word lists (I use Crossfire)
« Reply #3 on: June 19, 2020, 01:18:38 AM »
This guy's word list is listed on another thread... its very large and the guy has spent 10 years compiling and scoring it. Is it perfect?, no, and he knows it isn't, but its probably better than what you have, or anything you would create yourself in any short amount of time.

https://peterbroda.me/crosswords/wordlist/?fbclid=IwAR04CeR_nhEW5M7CoK6Pyc3lxtzAlD9i9nk6pYadGXWtWN9pTBNWvHCE2hk

When I click on the links at the bottom, it just opens a new tab with the words listed. all half a million+ of them. If that occurs for you, copy and paste into a .txt doc, save and rename the extension as .dict for Crossfire. Import and uncheck the existing lists in the Config>Dictionary area. One list to rule them all.

I too am wrapped up in making the perfect list, and the thing that I seem to be accomplishing the most in my word list compiling... is avoiding constructing crosswords. I have soo many lists, that Excel can't handle it. Working on a database solution at the moment. But I found this guy's scored list the other day and I probably should just give it up and start building crosswords with his list instead.

flight

  • Newbie
  • *
  • Posts: 38
Re: Understanding, obtaining, curating word lists (I use Crossfire)
« Reply #4 on: June 22, 2020, 04:57:53 PM »
I appreciate your input on the matter. I've already obtained that list and a few others. What eludes me, mostly, is understanding how to score words. I understand how to do it, I just don't really see much rhyme, reason, or consistency to it.

cubero

  • Newbie
  • *
  • Posts: 2
Re: Understanding, obtaining, curating word lists (I use Crossfire)
« Reply #5 on: June 30, 2023, 09:23:20 PM »
This guy's word list is listed on another thread... its very large and the guy has spent 10 years compiling and scoring it. Is it perfect?, no, and he knows it isn't, but its probably better than what you have, or anything you would create yourself in any short amount of time.

https://peterbroda.me/crosswords/wordlist/?fbclid=IwAR04CeR_nhEW5M7CoK6Pyc3lxtzAlD9i9nk6pYadGXWtWN9pTBNWvHCE2hk pizza tower

When I click on the links at the bottom, it just opens a new tab with the words listed. all half a million+ of them. If that occurs for you, copy and paste into a .txt doc, save and rename the extension as .dict for Crossfire. Import and uncheck the existing lists in the Config>Dictionary area. One list to rule them all.

I too am wrapped up in making the perfect list, and the thing that I seem to be accomplishing the most in my word list compiling... is avoiding constructing crosswords. I have soo many lists, that Excel can't handle it. Working on a database solution at the moment. But I found this guy's scored list the other day and I probably should just give it up and start building crosswords with his list instead.
Thank you for sharing the link to the word list compiled by Peter Broda.

bibyoung

  • Newbie
  • *
  • Posts: 1
Re: Understanding, obtaining, curating word lists (I use Crossfire)
« Reply #6 on: October 15, 2023, 11:05:03 PM »
This guy's word list is listed on another thread... its very large and the guy has spent 10 years compiling and scoring it. Is it perfect?, no, and he knows it isn't, but its probably better than what you have, or anything you would create yourself in any short amount of time.

https://peterbroda.me/crosswords/wordlist/?fbclid=IwAR04CeR_nhEW5M7CoK6Pyc3lxtzAlD9i9nk6pYadGXWtWN9pTBNWvHCE2hk connections

When I click on the links at the bottom, it just opens a new tab with the words listed. all half a million+ of them. If that occurs for you, copy and paste into a .txt doc, save and rename the extension as .dict for Crossfire. Import and uncheck the existing lists in the Config>Dictionary area. One list to rule them all.

I too am wrapped up in making the perfect list, and the thing that I seem to be accomplishing the most in my word list compiling... is avoiding constructing crosswords. I have soo many lists, that Excel can't handle it. Working on a database solution at the moment. But I found this guy's scored list the other day and I probably should just give it up and start building crosswords with his list instead.
If you have too many lists and cannot process them, please email them to me to store the information. I need it too.

juankax

  • Newbie
  • *
  • Posts: 1
Re: Understanding, obtaining, curating word lists (I use Crossfire)
« Reply #7 on: April 29, 2024, 04:13:47 AM »
thanks for the references

Glenn9999

  • Sr. Member
  • ****
  • Posts: 276
  • Common Solver, New Constructor
Re: Understanding, obtaining, curating word lists (I use Crossfire)
« Reply #8 on: April 29, 2024, 10:45:37 PM »
I'll try to help...

I don't understand the concept of ordering these dictionaries in the Preferences list. Why position one above or below another if they're all going to be used together in a single list (are they)?
On this concept of ordering dictionaries, think if you do something that's a special topic.  You might compile a list of baseball players and teams and stuff for doing baseball theme puzzles, and want that stuff to appear first.  Basically "use this, but if you can't use this, then use the next one".

Why keep the MyEdits.dict dictionary above the others (or not)?

The thought there is that you're going to prefer the words you added yourself over something general.

Do I need to merge dictionaries? What exactly happens when you merge dictionaries?

The basic idea here is that you'll have lists of words in disparate places which will be a lot slower to process.  So if you merge them, you can get one file with words that aren't duplicated and therefore is going to process better.  For myself, I collect/create word lists (about 35 MB right now) via my own processes.  (Frankly that's been more interesting to me than doing grids since I can code and have created processes like pulling data out of PUZ files.)  But I try to merge the generic word lists every once in a while too so I can have a universal file I can throw out.  I need a good way to knock down the OED words, basically the stuff people see as "junk fill" these days. (Though the OED list has been great for me trying to pattern match old grids.)

Why would I need to use the Edit-->Save Dictionary command? Which Dictionary does it save, how do I choose this (DO I even get to choose this?)?

I don't know much about Crossfire and this function, but the idea is to give you the ability to be able to add words and save them.

What are the ramifications of having multiple word lists wrapped into one (does this even happen?) when some are scored using 0-50 scale, others use 0-100, and some don't score at all? Should I merge these together? Won't that Frankenstein words of different scoring scales into each other and kinda render the whole point of using these scores moot, since it will all be unstandardized?

I think it orders preference based on the numbers, which will generate an issue if different word lists have different scoring ranges.  I don't score my word lists when I generate them, but the idea is that you assign the score to each of the words based on your preferences.  I haven't gotten into Crossword Compiler that much, but I'd prefer to see a list of possibilities and then actively select the word as opposed to auto-filling.

While completing fill, if I add words to a grid on the fly that aren't in "my dictionary", do these words automatically go into my dictionary or must I add them/save them? But what does "my dictionary" even mean, when I have multiple dictionary files to choose from?

I think you have to explicitly add them and save them.

What about if I want to eliminate all the scores from all the entries, must I do this one word at a time? In each different dictionary? Or can I do one big sweep and zero (or 100) them all out in one fell swoop? How do I know what score to give words that aren't scored, if I want to do so?

I don't know if there's a function in Crossfire that will remove all scores, but you can either strip them out via code or by knowledgeable use of a spreadsheet program.  I think in the raw version it's just CSV text, so you can just rename the extension to that and then load it into the spreadsheet program and delete the scores and save.  Or edit them if you want.  If you want to score words, you just give it a number based on your personal preference for seeing the word come up in your grid.

how do I properly set up and start using my dictionary(-ies?) and word lists from the ground up? How is a Dictionary different from a Word List (is it?)?

Go through the file and see what kind of words you like and don't like, score them appropriately according to your preferences, and go on.

Anyway, hope that helps some.

 


Powered by EzPortal