Wednesday 19 June 2013

Optimised Companion Planting / 3D Gardening Simulations

Updates (13/8/14):

The Effect of General Computing on Gardening?
Increases in the availability of general computing and processing power allow computers to be used to solve everyday problems where perhaps it was not considered necessary or required beforehand.
For quite a while I have thought that there must be a way to leverage available data online, 3d modelling software and information about companion planting relationships to help generate suggestions for gardening.

Garden Sim!
So whilst I have been experimenting with the following ideas whilst thinking about generating optimal planting strategies for gardens, I am not setting out to make the next version of Garden Simulator. But then again, I don't know what I am missing because I have never played it but come to think of it, I would rather actually do some real gardening, I mean- you cannot eat simulated fruit and vegetables which is, after all, the motivation for all this!
Given that general computing is ubiquitous and there is a lot of data available, it is worth investing some time to develop or experiment with simulation techniques even if only to generate new and interesting questions or spot opportunities for generating valuable insights for gardeners though a more targeted application of simulation for hard problems related to gardening / producing food effectively.

Simulation Overload?
It is important to realise the limitations of simulations. A professor in robotics once told me about the complexities in accurately and analytically modelling the motion of a moving rover with wheels taking into account wheel slippage and changing friction between different surfaces (not always clean, dust changing frictional constants etc). In the end it is just not practical to model the movement of the rover in this way and it is more beneficial to spend effort designing error-correcting feedback loops based on data from inertial navigation units (INUs). So at some point, simulations break down and become meaningless.

Empirical vs Analytical
Of course there is a lot to be said for adopting both empirical approaches and analytical approaches to this (and all) problems and the fact that there does exist analytically optimal solutions, does not diminish or replace the importance of 'local expertise' or other empirical bodies of knowledge. The problem is that 'local expertise' is not always available to be interrogated and expertise not always free.

Social
One of the most important things that a new gardener can do is connect with other (local) gardeners and share information for mutual benefit, but when just starting out it does not hurt to have a starting point like a computer-generated plan for a new vegetable plot rather than just a blank canvas. Then once you have this 'template', what better way to fill in the details that to consult a gardening 'hivemind' like stackexchange: http://gardening.stackexchange.com/

Toolbox
Here are some of the things I have been experimenting with in pursuit of these ideas:

What does the solution look like?
Firstly, the desired solution is a list of viable plants and some graphical representation of their optimal arrangement given some arbitrary input conditions.

Factors
It is important to think about about which factors might be significant in terms of the productivity of the garden which is, effectively, an artificially created natural system. Obviously the plants need the right weather conditions and nutrients in order to thrive and be productive which limits the selection of which species are sustainable i.e. require the least intervention / effort in order to be viable. In order to address this we often read gardening books or talk to neighbours in order to generate suggestions about what might work and what might not work.

Process
A lot of the fine detail and procedural knowledge gets left out of this process and we often end up with single solutions or a collection of disparate facts about gardening but not a complete system design / arrangement. I.e. nothing that is really actionable.

Context
I think it is important to establish the overall picture then pursue knowledge about its constituents in the proper context which means coming up with a plan or, at least, the beginnings of a plan before trying to leverage expertise on a specific topic.
 

We can intelligently select plants that are going to thrive in a given geographic area and a given time of year by looking at the shared results of trials from those who have tried before us. To the left there is a map showing the south east of england with a number of crowdsourced (variously around 8 to 9a) gardens which are in a particular USDA Hardiness Zone. This is a geographically defined area in which a specific category of plant life is capable of growing, as defined by climatic conditions, including its ability to withstand the minimum temperatures of the zone. Links to USDA Zone Gardens on myFolia:
1 2 2a 2b 3 3a 3b 4 4a 4b 5 5a 5b 6 6a 6b 7 7a 7b 8 8a 8b 9 9a 9b 10 10a 10b 11 11a 11b 12 12a 12b 13 13a 13b 14 
To the right is a temperature scale of hardiness zones, showing the average annual minimum temperature in degrees Celsius. The main factors determining average minimum temperature are elevation, latitude and proximity to the coast.

Given a set of input conditions like location, time of year, available soil types, soil pH, there must exist multiple optimal solutions to make the 'best' use of arbitrary amounts of available garden space. This result could be derived from comparing and contrasting all available gardening advice in published books and scientific literature about edible plant species (analytical) or it could be derived based on reports on what has worked and what has not worked from thousands of crowdsourced trials (empirical).

With sufficient levels of computing power and sophisticated techniques for making use of all that distilled knowledge, we can begin to provide optimal, informed, solutions to simple questions which are meaningful to people.

Why? Human beings need to eat!
What? should I plant? (plant species)

How? do I plant it? (plant spacing, techniques, techniques, particular details, tutorials and guides),
Where? perhaps somebody else technically owns the land but you can collaborate and share to make it possible to grow food.
When? gardening calendars can be drawn up to orgnise and plan.
Who?  Once list of plants is established it is possible to link to available crowdsourced data and also a network of people in order to help realise the intention of growing those plants. Seed swaps can be organised, information exchanged between people. Land / effort can be shared for mutual benefit.

Scraping Data about Plants with urllib in Python
Having generated a list of potential viable plants based on USDA zone, we can start to query datasets in order to get data about individual species.

Getting data about carrots:
values = {'query' : "carrot" }
data = urllib.urlencode(values)
version = choice(user_agents)
headers = { 'User-Agent' : version }
req = urllib2.Request("http://myfolia.com/plants/search", data, headers)
htmlText = urllib2.urlopen(req).read()
soup = BeautifulSoup(open(htmlText))
#need to parse the response and 'click' on first link in the results
Xpath to that top link: /html/body/div/div[2]/div/div/div/div[2]/ul/li/h2/p/a
which gives this: "plants/9-carrot-daucus-carota-subsp-sativus" path which should be combined with the base URL: "http://myfolia.com/" to give this link: http://myfolia.com/plants/9-carrot-daucus-carota-subsp-sativus

Then we need to scrape again, this time to targeting the plant data we are interested in.

Xpaths can be used to assist scraping:

name:      /html/body/div[2]/div[2]/div/div[2]/h1

latin name:    /html/body/div[2]/div[2]/div/div[2]/h2/em

height:    /html/body/div[2]/div[2]/div/div[3]/div[5]/div/form/ul/li[4]/label/span

width:    /html/body/div[2]/div[2]/div/div[3]/div[5]/div/form/ul/li[4]/label[2]/span

Once we have scraped all the data about plants we are interested in, we can then use that data to generate crude representations of the plants in 3D to assist geometric optimisation, identification and general arrangement:
cylinder(x, y, z, d, h, common_name, latin_name) {x,y,z} is the position in space.
cylinder(0, 0, 0, 2.5/2.54, 15/2.54, "orange", "Carrot", "Daucus carota subsp. sativus")     cylinder(15, 50, 0, 100/2.54, 120/2.54, "purple", "Rosemary", "Rosmarinus officinalis") cylinder(115, 100, 0, 120/2.54, 90/2.54, "green", "Courgette", "Cucurbita pepo var. cylindrica") cylinder(215, 150, 0, 50/2.54, 150/2.54, "red", "Tomato", "Solanum lycopersicum")
cylinder(315, 200, 0, 100/2.54, 50/2.54, "orange", "Potimarron", "Cucurbita maxima")
cylinder(415, 250, 0, 90/2.54, 30/2.54, "darkgreen", "Melon", "Cucumis melo")

generating cylinders with Python in Sketchup, representing mature plants based on crowsourced data scraped from myFolia
Abstraction
Understanding this problem at the appropriate level of abstraction given the techniques and the data available is required to actually provide something practically useful. Not all data is available (yet), not all manipulated variables / factors have as significant an output on the overall performance / output of the system and some intentional tweaking of the physical or chemical environment can take place in order to support an otherwise unsustainable but, nevertheless, desired species.
Whilst the last comment might seem like an endorsement to use fertilizers and automated irrigation systems, it is important to recognise that inventions ranging from clothing to central heating and insulation in homes have allowed species (homo sapiens) to thrive in unlikely places which would otherwise be inhospitable for us.

Breaking the Problem Down Further
  • 3D representation of the plants (ranging from simple cylinders with a radius and height, to complex and detailed 3D models).
  • 2D geometric optimisation algorithms and constraints
    • strategies to ensure optimal plant spacing
    • implications for maintenance, access to work on and tend to plants
    • best adherence to companion planting relationships:

    • reduction of monoculture / simple rows and drills of the same type of plant (as far as I know we dont tend to find neat rows of plants in nature, certainly not dense islands / clumps of only one type of plant so this cannot be optimal).
    • model as charged particles which attract and repel each other?
    • model as a free body diagram / series of interconnected springs + masses f = kx ?
  • which plants should be selected based on nutritional / aesthetic requirements of the gardener?
  • implications of the geographical location of the garden?
    • compbined with the time of year:
      • hours of daylight per day.
      • path of the sun over the garden.
        • local objects which shade the garden like trees or large shrubs
      • local climatic conditions and weather forecasts.
        • average rainfall and its distribution of rainfall throughout the year?
        • frost risk?
  • which other plants complement those selected / suggested?
    • which provide provide each other with complementary macronutrients?
      • rate of deposition of nitrates via nitrogen fixing nodules on roots (e.g. legumes)
  • which are responsible for localised inhibition of pests or other competing species of plant?
  • which provide competition for macronutrients, micronutrients?
  • Effects of size of plant and shadow casting / root growth of plants over their lifecycle which affects the casting of shadows and nutrient uptake and therefore have implications about orientation of the plot to the path of the Sun and inter-plant spacing.
  • desired ratio of species based on yield per plant
Having considered some or all of the above at varying levels of abstraction, it would be useful to output:
  • a diagram showing optimal spacing and juxtaposition of plants in the form of a 3D representation of the planting bed.
  • which can be exported in 2D Top / plan view.

  • A list of plants and planting schedule for reference in the garden.

Wednesday 5 June 2013

Cryptoparty.org Backup


Last night one of our members emailed the cryptoparty mailing list to say that the cryptoparty.org website was unexpectedly down. The site currently displays the text "Suspended due to spam."

It is a shame that people looking forward to attend cryptoparties this week who had publicised links to the website for their events will find that they are now broken.

Several members from the community, including myself, periodically scraped the cryptoparty.org website in order to ensure that everyone's contributions were not lost. My scrape was made with:

wget --recursive --no-clobber --page-requisites
--referer="http://www.google.com" --user-agent="Mozilla/5.0 (Windows;
U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"
--random-wait --html-extension --convert-links
--restrict-file-names=windows --domains cryptoparty.org --no-parent
https://cryptoparty.org/wiki/CryptoParty

Current List of cryptoparty.org mirrors/backups:

https://www.torservers.net/mirrors/cryptoparty.org/wiki/ (site as of 3rd June 2013)

http://www.samthetechie.com/cryptoparty.org/ (site as of 21st May 2013)

http://www.samthetechie.com/cryptoparty.org.tar.gz (backup, archived)

http://web.archive.org/web/*/http://cryptoparty.org/ (several snapshots over time:)