A contract-net model for foraging agents in dynamic environment

Öz In this paper we are interested to multi-agent foraging, where a team of agents search environment until they find some interested objects and return to nest. Apart from existing models for foraging, we consider a multi-agent system that the agents are restricted with fuel consumption. Additionally, interested objects in environments are in motion all the time. Then, the foraging problem is constrained with complex conditions of agents’ and the environment’s requirements. A Contract-Net model is proposed to solve this foraging considering all the constraints related with fuel consumption as well as dynamic environment conditions. Contract-Net is a well-known negotiation protocol where each agent (or node in the agents net) can be in role of a manager or contractor. Efficiency of the proposed model is investigated by some simulation experiments. Bu çalışmada biz çoklu etmen (yiyecek) arama sistemi ile ilgileniyoruz, ki bir ekip etmen çevrede bazı ilgi çeken nesneleri bulup yuva götürmeye kadar arama yapıyor. Arama için mevcut modellerin dışında, biz birçok etmen sistemi içinde etmenler yakıt tüketimi ile sınırlı olması durumu değerlendirmeyi göz önüne aldık. İlaveten, ilgi çeken nesneleri her zaman harket halinde. Bu durumda, arama sorunu etmenlere dayalı karmaşık koşullar ve çevre şartları ile sınırlanmıştır. Arama sorununu çözmek için bir Sözleşme-Ağ modeli yakıt tüketimi yanı sıra dinamik çevre koşulları ile ilgili tüm kısıtları göz önüne alarak önerilmiştir. Sözleşme-Ağ bir iyi tanınmış müzakere protokolüdür ki her etmenin (veya etmenler ağı içerisinde her düğümün) bir yönetici veya üstlenici rolü olabilir. Önerilen modelin verimliliği bazı simülasyon deneyleri ile incelenmiştir.


Introduction
Multi-agent systems (MASs) encounter with challenging issues such as task allocation, cooperation and coordination problems [1]- [3].In task allocation problem, a main approach is to decompose the task considering resources, and agents' capabilities.In the task decomposition, minimize communication and synchronization cost is desired [4].The decomposed task can be distributed among the agents through a market mechanism.Contract-Net is a popular market mechanism developed for multi agent systems by Smith [5].It is composed a chain of bids and awards run by two type of agents namely manager and contractor.An agent maybe gets either of manager's role and contractor's role for whole operation period or just for a portion of total time.The manager is the responsible agent to find a task that must be accomplished.Its responsibility is to announce the available task to the rest agents that are in contractor role.Then, contractors send bid over the announced task if they are interested to make cooperation into accomplishing the task.The manager investigates the bids according to a specific eligibility criterion and after a contract is formed with corresponding contractor(s).The contract specifies the task must be performed by which contractor agents.
In [6] a control system implemented using partition tasks through Contract-Net negotiation protocol.In [7] atomic contract types are proposed.Performance evaluation of Contract-Net for task accomplishment in time domain is investigated in [8].A threshold-limited load balance strategy for awarding agents is studied in [9].Optimal resource usage among a team of agents [10], cooperative control of multi-robots [11], task coordination on a physical multi mobile robot platform [12], cooperation among multiple vacuum cleaning robots [13], adaptive bidding using artificial neural networks for foraging robots [14], task allocation using game theory for multiple robots [15] are some studies related with Contract-Net based task allocation for multi-agent systems.
In this paper, we consider a Contract-net model for multiagent foraging problem.The goal is to develop a Contract-net model which can be implemented on a team of foraging agents that are commanded to collect some interested objects to a collection point (the nest).While a manager finds an interested object, it announces a "collectively transport task" to the team.Here, our developed model is basic communication platform between manager and the rest agents in order to deal with each other until a role assigned to the contractor agents to accomplish the transportation task.This paper supposes that the objects are big and therefore task decomposition is needed because the manipulation task is beyond the capabilities of any individual agent.In addition to, agents are restricted with limited fuel resources.The environment includes of the foraging team and objects, it is described a dynamic world.It is assumed that the objects are in motion along the environment.Therefore, developed Contract-Net model must meet the problem requirements and constraints such as restricted fuel resources, and the dynamics behavior of the environment (because of moving objects).However, a similar problem is studied in [16] for static environmental conditions.The rest of the paper is organized as the following.In second section, Contract-Net and multi-agent foraging problem are stated.In third section, simulation results are provided for a team composed of 10 agents.Conclusion described in the last section.

Contract-Net basics
Contract-Net provides a framework enable agents to decompose an available task to sub-tasks and most eligible agents are chosen to accomplish each sub-task.The agent that evaluates eligibility of interested agents to perform the subtasks is the manager.The rest agents are contractors, and whose showed their interests to perform a task through sending bids are potential contractors.Each agent can take a manager or a contractor role during whole foraging operation period.Also, an agent can be in manager role for a portion of whole time and then change its status to the contractor for rest of time or vice versa.
A manager has following responsibilities: define an available task, decompose it to sub-tasks, announce the sub-tasks to the potential contractors, receive and evaluation bides from the potential contractors, chose the most eligible contractors, award the winner agents with a Contract and finally receive and synthesize contractors' reports related with the sub-task accomplishment state from the winner agents.For a potential contractor the follow actions can be defined: receive the subtask announcements from the task manager, measure its capabilities for accomplish the announced sub-task, respond to the manager through declining the manager's demand or send a bid to it, receive the bid's answer (if he sent a bid), perform the sub-task physically if her bid won in manager side and finally make a report about the sub-task accomplishment results to the manager.
A sub-task announcement can be included following sections: ID's of potential contractors, the sub-task specifications, bid specifications and a deadline.The manager includes the ID's of the contractor agents who are in close distance to the task location.Manager is informed with the global positions of the contractor agents, the team size and each agent's ID.Also, each agent is equipped with a global sensor and communication system and it can announce its current position to the rest of team.A sub-task specification includes a brief description of the task, as well as maybe required resources or local environmental conditions.Bid specification includes the potential contractor capabilities to accomplish the announced sub-task, maybe includes information such as its fuel value, its current position, a value to show its interest rank to the subtask, etc. Deadline is a given time interval that the potential contractor should do a feedback to the manager with decline or bid message.If the manager receives no bid after the subtask deadline, the manager can make decision about change the sub-task or announcing of it again.

Foraging problem statement
A multi-agent foraging problem with dynamically changing world and restricted fuel resources is considered here.The world is assumed without any obstacle.Each agent can communicate with all rest agents in the world through a global communication infrastructure.The messages which can be communicated between agents (manager-contractor or contractor-manager) can be a sub-task announcement, a bid or decline responds and sub-task final report.The world is modelled as a two-dimensional square region.It is covered by a wall at the borders.Initially, all agents released randomly around borders in the world and they start to move randomly inside the environment.However, to get more effective result to explorer biggest part of the world, a "search algorithm" to cover maximum area can be employed in this step.Agents move inside the world to find some interested objects that already are defined for them.Here, the goal is to find an interested object and accomplish a collectively transportation to a collection/goal point.For a big size interested object, the manager needs to employ more than of one potential contractor agent.For small objects, a potential contractor is enough.If the manager isn't successful to establish a contract with any agent, then it leaves the role and will become a contractor who starts to accomplish the task by itself.
Assume a world with a proper size that contains  mobile agents.Initially, they are released around the four borders of the world (shown in green points in Figures).There is a collection/goal/nest point in the world.Also, there is a fuel station that it's located in a specific distance away from collection point.Fuel station coordinates informed to all agent, but each agent to refuel needs permission from a manager.Only manager agents are free to refuel always.The manager can reward contractor agents whom cooperated to accomplish a task with announcing the fuel station coordinates.There are some moving objects in the world (shown with black in Figures).Objects have their trajectory around the world and they can be transported by a single or by a few numbers of agents.Agents are not informed with the coordinates of the objects.Agents can detect the objects inside their sensory range.All agents are interested to collect objects to a goal point.It is assumed that agents move with higher speed respect to objects.Initially each agent  has a specific amount of fuel   .When an agent moves without load it consumes   unit per the base unit of length.If a single agent carries a small size object it would consume   unit of fuel per the base unit of length and if  agents cooperate in carrying a big size object the rate of fuel consumption for each agent will be decreased to   / unit per the base unit of length.
To encourage the agents to contribution in the transportation task, winner agents in the Contract-Net are awarded to refuel after accomplish the task.

Establishing contract-net
To solve the foraging problem through Contract-Net, we specify the components as follows.The first agent detects a moving object gets a manager role and moves along with the object.Simultaneously, the manager announces an available task (transportation task of the moving object to the goal point) to the rest of agents in the world.It is obvious that if the world is contained with more than of one object and some agents find separate objects in same time, therefore there are multiple managers where contractors can select to contribute which manager.Also, all agents in the environment are potential contractors at beginning.Based on some conditions related to the individual or environmental state, the agent maybe doesn't be interested to find an object and then it will not contribute in the search and/or transportation operation.This can be for example the reason of lake the fuel to motion in long distance.Such agent is not further a potential contractor.
Manager agent announces the task (or sub-task) to the society as (, , , ) where  is ID number of agent who is the manager ( = 1, . . ., ),  is label (ID number) of the moving object ( = 1, . . ., ) and (, ) is coordinates of the identified moving object.
After announcement of a task by a manager, some of potential contractor agents that are interested to contribute to accomplish of the task respond with an eligibility value to the manager.For a task announcement (, , , ) each potential contractor agent  calculates the following value as it eligibility, where    expresses the eligibility of agent  to accomplish the transportation task  of object ,   is the current fuel of agent ,    is the fuel amount that the agent receives from the fuel station if it contributes in the task  and    is the fuel amount consumes if the agent does the task .For each potential contractor  that    > 0 , it means that agent is interesting to accomplish the task and then it responds with a bid message to the manager.A contractor agent that isn't interested to a task (   > 0) responds with a decline message.A bid is specified as (, ,    ) where  and  demonstrates the agent and object ID's, respectively.Finally, the manager evaluates all received bids and chooses most eligible agents (with higher    ) for the task.In a given deadline, if the manager doesn't receive any bid message from the contractors, then the manager can decide to announce the task again or accomplish the task by itself.

Simulation results
In this section a simulation test setup is provided as follows to observe performance of the Contract-Net model.The Contract-Net model described in the previous section is designed to accomplish transportation task by contribution of some interested agents.We will have a sense of how well the model works.

Simulation world specifications:
We assume a simulation world with size 200×200 including of a goal point, a fuel station and one/two moving object(s) as it is shown in Figures 1 to 3. Also, 10 mobile agents are released in the word to do foraging.We assumed that the moving object(s) initially located in the middle of world.The fuel station and the collection/goal points are placed at a prescribed distance from each other as shown in Figures.The agents must find the moving object and catch it, then return the object to the goal point collectively or by single individual.The manager decides how many agents contribute to accomplish this task.Agents have a message mechanism to communicate with each other in order to accomplish multi-agent foraging.The world is also considered obstacle-free.The agents initially are released in the locations shown in Figures 1 to 3.
Additionally, parameters   and   are set to 0.05 and 0.5, respectively.For all agents initial fuel amount are set to   =500, ( = 1, . . ., ).When an agent detected the moving object and became the manager, it establishes a contract with some of rest agents to accomplish the transportation task.
Then, agents those have won the contract move toward the object with   time portion.At the end of each   time portion, a new task announcement is announced by the manager and a decision is made according to current environmental and task conditions such as current positions and current fuel amounts of the agents as well as current location of the object.This procedure is continued until the agents catch the object and start to transport it to the collection point (end of the task).It is obvious that more dynamic changing environments require smaller values for   .Agents can move one unit at each time step while the object moves half of one unit at each time step.We provide three simulation experiments as follows.In all simulations experiments we set   = 1 time steps.The contract refreshes at the end of each   time portion, then simulations are relatively slow but the generated paths (solutions) are near optimal.
Simulation Test 1: In first simulation experiment there is one moving object.It is assumed that there is only one moving object that it has its trajectory along a straight path.Initially, the object randomly takes one of the eight available directions (N, S, W, E, NW, NE, SW and SE) and starts motion along a straight path in the chosen direction whole time interval.Four snapshots of the experiment world in MATLAB corresponding to time steps 25, 75, 100 and 125 shown in Figure 1.It can be seen that agents 2, 4, 6 and 9 have approached the object and agents 2, 6 and 9 have caught it before step 125.Also, agents 5, 7, 8 and 10 have moved toward the object however, they have not been selected after some steps.Additionally, agents 1 and 3 have not moved toward the object at all.

Simulation Test 2:
In this experiment, we consider a world containing two moving objects.At the end of each time portion   , the objects randomly takes one of the directions (N, S, W, E, NW, NE, SW and SE) and continue movement along the chosen direction for the current   .Figure 2 shows four snapshots of this experiment.It is seen that the objects have randomly trajectories.As seen from Figure 2, agents 1, 2 and 3 have approached object 1 and agents 8 and 9 have approached object 2.
Simulation Test 3: In this experiment, the assumption is that both objects move along a straight path trajectory whole time interval.Figure 3 shows four snapshots of the experiment world where the objects travel in straight path.It is obvious agents 1, 2 and 3 have caught object 1 and agents 8 and 9 have caught object 2.
With using Contract-Net, manager agent decides based on received outputs of Eq. ( 1) from each potential contractor.Then, the parameters of Eq. ( 1) have the key effects in the foraging system performance.Here, we investigate the performance of the Contract-Net model in different aspects such as agents' participation rate and total fuel consumption rate inside three experimental setups.In Figure 4, we compare the rate of agents' participation in the foraging task in the three experiments carried out.The figure shows how many of agents of the multi-agent system cooperate to accomplish of the task during different time steps.As seen in beginning, the society in Experiment 1 has the maximum participation rate of agents to the foraging task, while in time step 60, it has the minimum participation rate.
Another useful comparison can be to measure fuel consumption amount during carrying out each experiment.In Figure 5, such a comparison for first 10 time steps is shown.As mentioned before, when an agent moves without load it consumes   unit per the base unit of length and here this parameter is set to   = 0.05.It is obvious that the multi-agent system used in Experiment 1 has the maximum total fuel consumption while the team in Experiment 2 has the minimum total fuel consumption.As a conclusion from Figures 4 and 5, multi-agent system used in Experiment 2 has the highest agents' participation rate as well as minimum fuel consumption rate in total time steps.Therefore, experimental setup 2 has the better foraging performance.

Conclusion
Multi-agent systems can be designed to using in a variety of applications.In this paper, we considered a foraging application where agents are restricted with fuel consumption.The agents communicate and contribute in transportation of interested objects based on Contract-Net model.Components of the Contract-Net to solve the foraging problem were devised while the limited fuel constraint effect in agent's decisions.By executing the proposed model in a loop in every tfr seconds, the method was simulated for dynamic environments contain multiple moving objects.It is observed that generated solutions by the model are successful to accomplish the foraging task in a given total time steps.We showed performance of our Contract-Net model in different aspects such as agents' participation rate and total fuel consumption rate inside three experimental setups.Results showed that experimental setup 2 has the better foraging performance.

Acknowledgment
This work is done under the research project entitled "Solving of transportation problem in foraging agents using Contract-Net algorithm" funded by Islamic Azad University, Ahar Branch, Iran.
Each agent , ( = 1, … , ) know its current position that shown as   = (  ,   ).Positions of objects are time-varying and coordinates of each object , ( = 1, … , ) shown with   = (  ,   ), where  is the number of objects.The goal point (collection point) coordinates shown as   = (  ,   ) and all agents are informed with   .Fuel station coordinates shown as   = (  ,   ).

Figure 4 :
Figure 4: Comparison the rate of agents' participation in the foraging task in the three experiments carried out (Size of society, i.e. number of agents versus time step).

Figure 5 :
Figure 5: Comparison of total fuel consumption amount during carrying out each experiment.